fauna-shell
fauna-shell copied to clipboard
fauna
New fauna-shell only supports queries up to ~100KB
I think fauna supports query sizes up to several megabytes, so this is a shell/driver bug:
davidgriffin@skadi flotsam % fauna --version
4.0.0-beta
davidgriffin@skadi flotsam % cat big_97k_query.fql| fauna query --profile DevDemoDB -
{
...
}
davidgriffin@skadi flotsam % cat big_102k_query.fql| fauna query --profile DevDemoDB -
... help text ...
An unexpected error occurred...
EAGAIN: resource temporarily unavailable, read
If you believe this is a bug, please report this issue on GitHub: https://github.com/fauna/fauna-shell/issues
Running with --verbosity=9 the 102k query actually succeeds, but if I throw a 1MB file at it...
davidgriffin@skadi flotsam % cat big_1M_query.fql| fauna query --verbosity=9 --profile DevDemoDB -
... help text ...
An unexpected error occurred...
EAGAIN: resource temporarily unavailable, read
If you believe this is a bug, please report this issue on GitHub: https://github.com/fauna/fauna-shell/issues
[error]: EAGAIN: resource temporarily unavailable, read
at readFileSync (node:fs:448:20)
at resolveInput (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:96814:12)
at Object.queryCommand [as handler] (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:96824:22)
at /usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80016:46
at maybeAsyncResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:79818:83)
at CommandInstance.handleValidationAndGetResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80015:19)
at CommandInstance.applyMiddlewareAndGetResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80046:242)
at CommandInstance.runCommand (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:79953:212)
at [runYargsParserAndExecuteCommands] (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:82735:91)
at YargsInstance.parse (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:82072:59)
I'm loving the new CLI so keep up the good work!
Full verbose=9 output...
davidgriffin@skadi flotsam % cat big_1M_query.fql| fauna query --verbosity=9 --profile DevDemoDB -
[config]: Found default config file named "fauna.config.yaml". Using it.
[config]: Reading config from fauna.config.yaml.
[config]: Using profile DevDemoDB...
[config]: Applying config: {
"color": true,
"secret": "REDACTED"
"url": "https://db.fauna-dev.com"
}
[config]: Found default config file named "fauna.config.yaml". Using it.
[config]: Reading config from fauna.config.yaml.
[config]: Using profile DevDemoDB...
[config]: Applying config: {
"color": true,
"secret": "REDACTED",
"url": "https://db.fauna-dev.com"
}
[argv]: {
"_": [
"query"
],
"verbosity": 9,
"profile": "DevDemoDB",
"p": "DevDemoDB",
"root": "/Users/davidgriffin/IdeaProjects",
"ops": "/Users/davidgriffin/IdeaProjects/ops",
"util": "/Users/davidgriffin/IdeaProjects/ops/util",
"bin": "/Users/davidgriffin/IdeaProjects/ops/bin",
"color": true,
"secret": "fnAF********************************P9X7",
"url": "https://db.fauna-dev.com",
"config": ".",
"json": false,
"quiet": false,
"verbose-component": [],
"verboseComponent": [],
"user": "default",
"u": "default",
"local": false,
"api-version": "10",
"v": "10",
"apiVersion": "10",
"format": "fql",
"f": "fql",
"timeout": 5000,
"performance-hints": false,
"performanceHints": false,
"include": [
"summary"
],
"account-url": "https://account.fauna.com",
"accountUrl": "https://account.fauna.com",
"$0": "fauna",
"fql": "-"
}
[argv]: Existing Fauna environment variables: {"FAUNA_ROOT":"/Users/davidgriffin/IdeaProjects","FAUNA_OPS":"/Users/davidgriffin/IdeaProjects/ops","FAUNA_UTIL":"/Users/davidgriffin/IdeaProjects/ops/util","FAUNA_BIN":"/Users/davidgriffin/IdeaProjects/ops/bin"}
[argv]: reading query from stdin
[error]: unknown error thrown: Error
[error]: EAGAIN: resource temporarily unavailable, read
at readFileSync (node:fs:448:20)
at resolveInput (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:96814:12)
at Object.queryCommand [as handler] (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:96824:22)
at /usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80016:46
at maybeAsyncResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:79818:83)
at CommandInstance.handleValidationAndGetResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80015:19)
at CommandInstance.applyMiddlewareAndGetResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80046:242)
at CommandInstance.runCommand (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:79953:212)
at [runYargsParserAndExecuteCommands] (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:82735:91)
at YargsInstance.parse (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:82072:59) {
[stack]: [Getter/Setter],
[message]: 'EAGAIN: resource temporarily unavailable, read',
errno: -35,
code: 'EAGAIN',
syscall: 'read'
}
fauna query [fql]
Run an FQL query.
API:
-u, --user CLI user to run the command as. You must first log in as the user using 'fauna login'. [string] [default: "default"]
--local Use a local Fauna container. Sets --url to 'http://0.0.0.0:8443' and --secret to 'secret'. [boolean] [default: false]
--url URL for Core HTTP API requests made by the command. Defaults to https://db.fauna.com. [string]
--secret Secret used for authentication. Can't be used with --database or --role. [string]
--account-key Fauna account key used for authentication. Can't be used with --user or --secret. [string]
-d, --database Database, including Region Group and hierarchy, to run the command in. Ex: 'us/my_db', 'eu/parent_db/child_db', 'global/db'. Can't be used with --secret. [string]
-r, --role Role used to run the command. Can't be used with --secret. [string]
-v, --api-version FQL version to use. [string] [choices: "4", "10"] [default: "10"]
-f, --format Output format for the query. When present, --json takes precedence over --format. Only applies to v10 queries. [string] [choices: "fql", "json"] [default: "fql"]
--typecheck Enable typechecking. Defaults to the typechecking setting of the database. [boolean]
--timeout Maximum query runtime in milliseconds. Only applies to v10 queries. [number] [default: 5000]
--performance-hints Output performance hints. Sets --include summary. Only applies to v10 queries. If no performance hints are returned, no hints are output. [boolean] [default: false]
Positionals:
fql FQL query to run. Use - to read from stdin. [string]
Output:
--color Enable color formatting. Use --no-color to disable. [boolean] [default: true]
--json Output the results as JSON. [boolean] [default: false]
--quiet Suppress all log messages except fatal errors. Overrides --verbosity and --verbose-component. [boolean] [default: false]
Config:
--config Path to a CLI config file to use. If provided, you must specify a profile. [string] [default: "."]
-p, --profile Profile from the CLI config file to use. [string]
Debug:
--verbose-component Components to emit logs for. Overrides the --verbosity flag. Pass values as a space-separated list. Ex: --verbose-component fetch error.
[array] [choices: "argv", "completion", "config", "creds", "error", "fetch"] [default: []]
--verbosity Least critical log level to emit. Accepts 1 (fatal) to 5 (debug). Lower values represent more critical logs. [number] [default: 0]
Options:
-h, --help Show help. [boolean]
--version Show the Fauna CLI version. [boolean]
--include Query response info to output. Pass values as a space-separated list. Ex: --include summary queryTags.
[array] [choices: "all", "none", "txnTs", "schemaVersion", "summary", "queryTags", "stats"] [default: ["summary"]]
-i, --input Path to a file containing an FQL query to run. [string]
-o, --output Path to a file where query results are written. Defaults to stdout. [string]
Examples:
fauna query "Collection.all()" --database us/my_db Run the query in the 'us/my_db' database and write the results to stdout.
fauna query "Collection.all()" --database us/my_db --role server Run the query in the 'us/my_db' database using the 'server' role.
fauna query "Collection.all()" --secret my-secret Run the query in the database scoped to a secret.
fauna query -i /path/to/query.fql --database us/my_db Run the query from a file.
echo "1 + 1" | fauna query - --database us/my_db Run the query from stdin.
fauna query -i /path/to/query.fql --output /tmp/result.json --database us/my_db Run the query and write the results to a file.
An unexpected error occurred...
EAGAIN: resource temporarily unavailable, read
If you believe this is a bug, please report this issue on GitHub: https://github.com/fauna/fauna-shell/issues
[error]: EAGAIN: resource temporarily unavailable, read
at readFileSync (node:fs:448:20)
at resolveInput (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:96814:12)
at Object.queryCommand [as handler] (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:96824:22)
at /usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80016:46
at maybeAsyncResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:79818:83)
at CommandInstance.handleValidationAndGetResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80015:19)
at CommandInstance.applyMiddlewareAndGetResult (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:80046:242)
at CommandInstance.runCommand (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:79953:212)
at [runYargsParserAndExecuteCommands] (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:82735:91)
at YargsInstance.parse (/usr/local/lib/node_modules/fauna-shell/dist/cli.cjs:82072:59)
This is the offending piece of code: https://github.com/fauna/fauna-shell/blob/main/src/commands/query.mjs#L71
Most likely, reading queries from stdin will probably need to use process.stdin.read()instead. We should consider upper limits to what is read into memory. And it all needs to be read into memory before it can be sent to Fauna.
This is the offending piece of code: https://github.com/fauna/fauna-shell/blob/main/src/commands/query.mjs#L71
Most likely, reading queries from stdin will probably need to use
process.stdin.read()instead. We should consider upper limits to what is read into memory. And it all needs to be read into memory before it can be sent to Fauna.
Cool, it looks like the stated limit on the server side is 16MB, so I think that's what we want the driver/shell to support.