goodtables.io
goodtables.io copied to clipboard
Clean the codebase
Overview
After MVP we need to clean our codebase a little bit removing not used parts etc
Tasks
- [x] remove
result_backend
if it's not needed by chains - http://docs.celeryproject.org/en/latest/userguide/canvas.html - this one is imporants because it leads to additonal load - [x] remove examples?
- [x] remove commands?
- [x] remove tox.ini?
- [x] remove setup.cfg?
- [x] remove
goodtablesio.VERSION
? - [x] update readme
- [x] improve naming (helpers etc)?
- [x] signals to separate module?
- [x] better approach than global
tasks_db
? - [x] import consistency
- [x] docstrings
- [ ] decide between having
setup.py
ORrequirements.dev.txt
- [x] ~~merge concepts of validation_conf and job_conf to jobconf (raw as goodtables.yml)?~~
- [x] ~~tests to root?~~
- [ ] codestyle
- [ ] remove functions wrapping sqlalchemy models (user, job) (?)
- [ ] review TODOs
- [x] rename goodtables.io to backend? (MOVED #158)
- [x] fix coverage (back + front) (MOVED #165)
- [x] remove Procfile, runtime.txt
- [ ] rename
validation_conf
toinspection_conf
(also taskvalidate
->inspect
) - it's howgoodtables-py
names it (less confusion) + no conflict with json schema validation (?) - [ ] review how we use celery!!! there is some weak points like non atomic write to db/task run, potential hanging etc (checks with @roll)
- [ ] clean
public
dir before frontend building - [ ] use
jquery.js
via npm? - [ ] use
bootstrap.js
via npm? - [ ] review contents of the bundle.js (if needed - bootstrap and styles is inside so the reason of big size)
- [ ] be consistent with github naming. I suppose it should be
github/repo/<owner>/<repo>
(we use org instead of repo in some places which is not correct) - [ ] why we use
success/failure
insteadvalid/invalid
across codebase, database and UI - it doesn not seem really correct - [ ] we should properly process first github hook request after activation (it has dummy payload but in GitHub UI it's shown as a hook failure) - we have just ignore it and return 200
- [ ] should we catch expected error inside validation tasks to do not have
failed
task in http://flower.goodtables.io/tasks? - [ ]
job.source_id
should be NOT NULL
For now, putting this in beta, not alpha. It is too early to have a dedicated refactoring tasks IMHO. Of course, some of these could get picked up while doing other tasks.
@pwalsh All of them are pretty minor. Renaming, deleting unused things, moving functions etc. No coding here. Like a few hours of work for the whole task.
Still recommend to do it as soon as possible (when @amercader will be having time for it) because some messy parts of codebase are slowing down the development.
@amercader I'm not to fussed about this for Beta, but I assume you and @roll just fix what comes up as you touch parts, so leaving it here of course with assignment to each of you.
@amercader We could finish it straight after the Beta release. There are many minor things.