tornadoweb
autoreload: Pre-reload file change hook (syntax checking, etc)
not sure how to add this to code. maybe someone to add this to project. this is the first time i wrote python
import os
import tornado.ioloop
import tornado.web
import tornado.autoreload
import time
from datetime import datetime
import json
compileerr=""
class MainHandler(tornado.web.RequestHandler):
def get(self):
global compileerr
if compileerr!="":
self.write("<pre>Compile error:\n"+compileerr);
#self.render("index.html")
else:
try:
with open(os.path.join(os.path.dirname(__file__), 'index.html')) as f:
self.write(f.read())
except IOError as e:
self.write("404: Not Found")
es_empty_result={ "hits": {"hits": [], "total": 0, "max_score": None}, "_shards": {"successful": 5, "failed": 0, "total": 5 }, "took": 3, "timed_out": False }
#some apis
class List1Handler(tornado.web.RequestHandler):
def get(self):
global es_empty_result
id=self.get_argument("id", default=None, strip=False)
user=self.get_argument("user", default=None, strip=False)
self.write(json.dumps(es_empty_result, sort_keys = False, indent = 4))
class List2Handler(tornado.web.RequestHandler):
def get(self):
global es_empty_result
id=self.get_argument("id", default=None, strip=False)
user=self.get_argument("user", default=None, strip=False)
self.write(json.dumps(es_empty_result, sort_keys = False, indent = 4))
application = tornado.web.Application([
(r"/$", MainHandler),
(r"/list1$", List1Handler),
(r"/list2$", List2Handler),
(r"/.*\.py$", MainHandler),# don't serve python as static files
(r"/(.*)$", tornado.web.StaticFileHandler, dict(path=os.path.dirname(__file__))),
])
if __name__ == "__main__":
import py_compile
import logging
from threading import Timer
def ontimer_reload(prev_mtime,filepath,mainfile): # wait for file to stop changing, than check syntax,than reload
global compileerr
statinfo=os.stat(filepath)
if statinfo.st_size>0 and statinfo.st_ctime==statinfo.st_mtime and statinfo.st_mtime-prev_mtime==0:
compiled=False
try:
py_compile.compile(mainfile,doraise=True)
compiled=True
except py_compile.PyCompileError as e:
compileerr=str(e)
print "reloading: compile error..."
print e;
if compiled:
logging.info("%s modified; restarting server", filepath)
tornado.autoreload._reload()
else:
print "reloading: waiting for file upload complete."
Timer(0.3, ontimer_reload,(statinfo.st_mtime,filepath,mainfile)).start()
#usage:Timer(0.3, ontimer_reload,(path,__file__)).start()
def new_check_file(modify_times, filepath):
try:
modified = os.stat(filepath).st_mtime
except Exception:
return
if filepath not in modify_times:
modify_times[filepath] = modified
return
if modify_times[filepath] != modified:
modify_times[filepath] = modified
Timer(0.3, ontimer_reload,(modified, filepath,__file__)).start()
#monkey patch tornado reload class
tornado.autoreload._check_file = new_check_file
print "server start"
application.listen(8888)
#def beforereloading():
# print "reloading: exiting..."
#tornado.autoreload.add_reload_hook(beforereloading)
#tornado.autoreload.watch(os.path.abspath('./file')) #additional file to watch
tornado.autoreload.start()
tornado.ioloop.IOLoop.instance().start()
tornado.autoreload.wait()
"""
paste large text here to test like duplicate the folowing text until 1000 lines
HOWTO'S AND FAQ' Difference between mtime, ctime and atime
A common mistake is that ctime is the file creation time. This is not correct, it is the inode/file change time. mtime is the file modification time. A often heard question is What is the ctime, mtime and atime?.This is confusing so let me explain the difference between ctime, mtime and atime.
ctime
ctime is the inode or file change time. The ctime gets updated when the file attributes are changed, like changing the owner, changing the permission or moving the file to an other filesystem but will also be updated when you modify a file.
mtime
mtime is the file modify time. The mtime gets updated when you modify a file. Whenever you update content of a file or save a file the mtime gets updated.
Most of the times ctime and mtime will be the same, unless only the file attributes are updated. In that case only the ctime gets updated.
atime
atime is the file access time. The atime gets updated when you open a file but also when a file is used for other operations like grep, sort, cat, head, tail and so on.
"""
just a note, in tornado, you can avoid $ at the end, it is automatically added, not like Django ^_^
https://github.com/tornadoweb/tornado/blob/master/tornado/web.py#L2779
you can try to add the feature here: https://github.com/tornadoweb/tornado/blob/master/tornado/autoreload.py
and welcome to python, you will like it, espetially Tornado :D
We already have a solution for the SyntaxError problem: prefix your command with "python -m tornado.autoreload". This handles not just the syntax errors that can be detected with py_compile, but also ImportErrors, NameErrors, and other startup-time problems that might prevent the app from getting into its own reload loop.
A delay before restarting is an interesting idea. Since you mention uploads it sounds like you don't need it to allow any in-progress work in the server to finish, but to wait for some file sync operation to finish. In this case I think you can accomplish it by adding a reload hook that simply calls time.sleep.
Thank you @abdelouahabb :)
I wanted it to respond faster and correctly also I wanted to display somehow the error to the web.
I did a reload hook with time.sleep 1 second . It generally works but sometimes fails and fails on error even with python -m,( if it was working i was not searching for a solution or developing a solution for this). I don't know what i am doing wrong but it is not working - i am using python -m tornado.autoreload and it still crashes on errors, but you said it expected to not crash :
#tornado.autoreload._check_file = new_check_file
def beforereloading():
time.sleep(0.5) #intentionally so it will be short on upload
print "reloading: exiting..."
tornado.autoreload.add_reload_hook(beforereloading)
root@server1:/app/test# python -m tornado.autoreload index.py
server start
reloading: exiting...
server start
reloading: exiting...
File "index.py", line 243
self.write(json.dumps({"question":q_re,"byquestion":by_q_re,"bysitu
^
SyntaxError: EOL while scanning string literal
root@server1:/app/test#
the problem is about the end of line, in python there is no braces, python uses end of lines (CR LF), so if you dont want to end an instruction you can use a comma to go to a new line, for readability for example:
# This will not return an error
example = {'hello':34, 'how are you': 56,
'nice'=555}
#This has an error
example = {'hello':34, 'how are you': 56
'nice'=555}
@abdelouahabb i haven't explained well. this happens during upload, my code is ok.
sorry, i saw the
self.write(json.dumps({"question":q_re,"byquestion":by_q_re,"bysitu
:D
There is a race condition in the command-line version of autoreload - if a file changes immediately after it's been loaded with a syntax error, that version will be seen as the "original" by autoreload so the file will have to change a second time to trigger a reload. There's not an elegant way to fix this because of the way that autoreload doesn't know what files you're interested in until they have been loaded.
It's probably best to address this problem outside of the autoreload system. Either make sure that whatever file syncing system you're using uses atomic writes (for example, rsync does this but scp does not), or explicitly restart the server when the sync is finished instead of relying on autoreload.
It's probably best to address this problem outside of the autoreload system.
FWIW, I no longer believe this. There are plenty of ways that an incomplete file can be written (VCS merge conflicts, editor auto-saves, etc), and it's not reasonable to forbid them all. Some sort of pre-reload hook to do a sanity check of the modified file makes sense. It's just a question of designing the right interfaces.