harelba
Feature request: 'FROM'-specific command-line parameters
Something like that:
q "SELECT A,B FROM(-H -d,) qqq.csv WHERE B NOT IN (SELECT c1 FROM(-d@ -c1) lines.txt)"
Yeah, this is definitely a required feature, although I'm not sure of how it's best to implement it. Two things - one is that I'm not sure it should clutter the query itself or be part of the parameters to q, and the other is that adding it to the query itself requires a full blown sqlite3 parse tree generator, which I haven't found a good one yet in python which actually does the job (Having such a parse tree generator would allow much more good stuff to happen).
Also, I'm not sure about the frequency that this kind of problem happens to users. I'm planning on adding a "feedback mode" to q, which will allow to send feedback to my email directly from the command line. Maybe then I'll have more stats regarding what is more frequent and what isn't.
Another approach: decouple table name and file name, makeing the former only a default for the latter.
Example syntax:
q -O 'SELECT * FROM a LEFT JOIN b ON a.qqq = b.c1' -d, -H a=a.csv -d: b=/etc/passwd
Before the query there are input parameters for implicit (non-overridden) files.
I've created a new python-level API for q, which will allow q to be used from inside python code, as a module. The API level fully supports per-file input parameters. Still not sure how to expose it to the command line level, though, so we'll see how to expose it.
An alpha version of the new API will be committed into the master branch in a couple of days. Any feedback on it will be greatly appreciated.
Harel
Alpha branch of the python api has been committed to https://github.com/harelba/q/tree/expose-as-python-api.
Any input would be helpful and appreciated.
Harel
I think the main advantage of q is ability to use it as a one-liner, without any "environment". I don't want new features (this issue, the caching) to be API-only and command-line interface to "stagnate". q should support complex things, but it's primary goal should be doing simple things really simply.
For-programmer q analogue can be a bit like F#'s type providers, bringing information about rows in a CSV file into tab completion in Ipython or an IDE. Such library can be a backbone for q, not a "mode" of q.
The's definitely the idea :) the python API is only meant to be a backend for the command line - Nothing will change from the command line point-of-view.
I'll try to provide a proper way to provide separate parameters for each file in the command line.
On Sun, Nov 23, 2014 at 9:10 PM, Vitaly Shukela [email protected] wrote:
I think the main advantage of q is ability to use it as a one-liner, without any "environment". I don't want new features (this issue, the caching) to be API-only and command-line interface to "stagnate".
For-programmer q analogue can be a bit like F#'s type providers http://msdn.microsoft.com/en-us/library/hh156509.aspx, bringing information about rows in a CSV file into tab completion in Ipython or an IDE. Such library can be a backbone for q, not a "mode" of q.
— Reply to this email directly or view it on GitHub https://github.com/harelba/q/issues/70#issuecomment-64145802.
Do you like my idea of the interface for this? (with a=a.csv and such, see above).
It is:
- More or less intuitive;
- Backwards-compatible;
- Allows untangling the possibly complex filename (with spaces, backslashes, newlines, non-printable characters and so on) from the query line.
@vi I believe that the notion of your idea of providing aliases to the files, along with the relevant modifiers would work well.
Once the python API is commented upon and finalized, I will get to exposing that on the command line.
@vi The python API is now the driver for the newly released 1.5.0 version, and I have a development branch which describes the API. Your input will be welcome. https://github.com/harelba/q/blob/expose-as-python-api/README.markdown