sqlalchemy backend

[coh2]

that way people could use whatever database they like, which allows better integration for them.

the sqlite backend should stay around for the quick+dirty jobs.

[9b]

I like the idea. Was that the general thought behind the generic storage class?

[coh2]

not entirely but to a degree yes as that will allow anyone to add a custom storage backend with ease.

[9b]

I like the concept. To some degree however, it seems we need to have a set of supported DBs or an easy way for integration not unless we off load this to a user. One of the issues that comes to mind is how a SQL database will differ from a NoSQL database query language.

[coh2]

that's where the abc kicks in, as with it, we can define a set of functionality we need and leave it to the backend to do whatever it takes in order to implement it.

for the current prime example, see already_visited_url in sqlite_backend.py vs mongodb_backend.py -- same functionality and API to our stuff, yet totally differing backend types and APIs.

[coh2]

head up: done with the table layout (except for testing when the backend is done), half of the backend implementation remains. Will likely be another week(end) until it is fully ready.