goodtables.io icon indicating copy to clipboard operation
goodtables.io copied to clipboard

Clean the codebase

Open roll opened this issue 7 years ago • 4 comments

Overview

After MVP we need to clean our codebase a little bit removing not used parts etc

Tasks

  • [x] remove result_backend if it's not needed by chains - http://docs.celeryproject.org/en/latest/userguide/canvas.html - this one is imporants because it leads to additonal load
  • [x] remove examples?
  • [x] remove commands?
  • [x] remove tox.ini?
  • [x] remove setup.cfg?
  • [x] remove goodtablesio.VERSION?
  • [x] update readme
  • [x] improve naming (helpers etc)?
  • [x] signals to separate module?
  • [x] better approach than global tasks_db?
  • [x] import consistency
  • [x] docstrings
  • [ ] decide between having setup.py OR requirements.dev.txt
  • [x] ~~merge concepts of validation_conf and job_conf to jobconf (raw as goodtables.yml)?~~
  • [x] ~~tests to root?~~
  • [ ] codestyle
  • [ ] remove functions wrapping sqlalchemy models (user, job) (?)
  • [ ] review TODOs
  • [x] rename goodtables.io to backend? (MOVED #158)
  • [x] fix coverage (back + front) (MOVED #165)
  • [x] remove Procfile, runtime.txt
  • [ ] rename validation_conf to inspection_conf (also task validate -> inspect) - it's how goodtables-py names it (less confusion) + no conflict with json schema validation (?)
  • [ ] review how we use celery!!! there is some weak points like non atomic write to db/task run, potential hanging etc (checks with @roll)
  • [ ] clean public dir before frontend building
  • [ ] use jquery.js via npm?
  • [ ] use bootstrap.js via npm?
  • [ ] review contents of the bundle.js (if needed - bootstrap and styles is inside so the reason of big size)
  • [ ] be consistent with github naming. I suppose it should be github/repo/<owner>/<repo> (we use org instead of repo in some places which is not correct)
  • [ ] why we use success/failure instead valid/invalidacross codebase, database and UI - it doesn not seem really correct
  • [ ] we should properly process first github hook request after activation (it has dummy payload but in GitHub UI it's shown as a hook failure) - we have just ignore it and return 200
  • [ ] should we catch expected error inside validation tasks to do not have failed task in http://flower.goodtables.io/tasks?
  • [ ] job.source_id should be NOT NULL

roll avatar Nov 30 '16 16:11 roll

For now, putting this in beta, not alpha. It is too early to have a dedicated refactoring tasks IMHO. Of course, some of these could get picked up while doing other tasks.

pwalsh avatar Dec 05 '16 07:12 pwalsh

@pwalsh All of them are pretty minor. Renaming, deleting unused things, moving functions etc. No coding here. Like a few hours of work for the whole task.

Still recommend to do it as soon as possible (when @amercader will be having time for it) because some messy parts of codebase are slowing down the development.

roll avatar Dec 05 '16 08:12 roll

@amercader I'm not to fussed about this for Beta, but I assume you and @roll just fix what comes up as you touch parts, so leaving it here of course with assignment to each of you.

pwalsh avatar Jan 27 '17 07:01 pwalsh

@amercader We could finish it straight after the Beta release. There are many minor things.

roll avatar Apr 18 '17 09:04 roll