jq
jq copied to clipboard
jqlang
Edit files in place
With GNU Sed for example you can edit files in place:
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if SUFFIX supplied)
However I see no option for this with JQ, currently I must do this:
br=$(mktemp)
jq .charlie delta.json > "$br"
mv "$br" delta.json
Here is a workaround, Awk slurp:
jq .firstName alfa.json | awk 'BEGIN{RS="";getline<"-";print>ARGV[1]}' alfa.json
Scripting around this is not an option? In shell:
tmpf=$(mktemp --tmpdir=$(dirname $f) -t)
jq ... "$f" > "$tmpf"
mv -f "$tmpf" "$f"
By analogy: Would it be acceptable if <insert your favorite word processor here> made you save to a different file every time you wanted to save your document?
It's not that the problem can't be worked around, it's that doing so is cumbersome. The fact that something as fundamentally stream oriented as sed has an in-place option is a pretty good indicator that this is common expectation/desire.
I see. I've generally not bothered with sed's -i option. For me the problem has been two-fold: a) it is (well, was) not portable enough, b) I always have to check whether such a tool's in-place update functionality is documented as working this way (rename into place), because I generally need atomicity. And then there's Windows (where renames generally don't work when there's open file handles for the file in question, unless all the processes that opened it used an option to allow it).
On Mon, Jun 9, 2014 at 10:23 AM, ipmcc [email protected] wrote:
By analogy: Would it be acceptable if made you save to a different file every time you wanted to save your document?
It's not that the problem can't be worked around, it's that doing so is cumbersome. The fact that something as stream oriented as sed has an in-place option is a pretty good indicator that this is common expectation/desire.
— Reply to this email directly or view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-45502984.
Anyways, I can add this easily. My only concern is to limit the number of new command-line options. Since this option couldn't be implemented in a jq program, I'll probably do it. Though perhaps a generic utility to wrap filters with might be better:
$ inplace some-file command args
which would run the given command with some-file as the standard input
and which would redirect the command's output to a temporary file in the
same directory then rename that into place. That's the Unix philosophy,
after all.
On Mon, Jun 9, 2014 at 10:58 AM, Nico Williams [email protected] wrote:
I see. I've generally not bothered with sed's -i option. For me the problem has been two-fold: a) it is (well, was) not portable enough, b) I always have to check whether such a tool's in-place update functionality is documented as working this way (rename into place), because I generally need atomicity. And then there's Windows (where renames generally don't work when there's open file handles for the file in question, unless all the processes that opened it used an option to allow it).
On Mon, Jun 9, 2014 at 10:23 AM, ipmcc [email protected] wrote:
By analogy: Would it be acceptable if made you save to a different file every time you wanted to save your document?
It's not that the problem can't be worked around, it's that doing so is cumbersome. The fact that something as stream oriented as sed has an in-place option is a pretty good indicator that this is common expectation/desire.
— Reply to this email directly or view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-45502984.
Ah, there's a sponge(1) that almost works this way:
http://joeyh.name/code/moreutils/ http://backreference.org/2011/01/29/in-place-editing-of-files/
I think both, sponge(1) and my idea of an inplace(1) would be very useful generally and very much in keeping with the Unix philosophy.
Would that work for you?
@svnpenn @ipmcc @karelv
Regardless of how you feel about this issue, what think ye of this:
https://github.com/nicowilliams/inplace
?
Is the -i option available or not? In the linked commit it says that this feature was removed.
@drorata - It was (as best I can tell) never available in any official jq release, and is certainly not available in jq 1.4 or 1.5. It was added as an experimental feature on Jul 20, 2014, and removed on Mar 5, 2015.
@svnpenn - Although there are obviously circumstances when -i would be convenient, jq is more like awk than sed. The earlier attempt to endow jq with -i confirms that such attempts will be fraught with difficulties. awks do not have an -i option, and I think it is asking too much of the jq developers/maintainers to devise and implement such an option, especially given there are easy workarounds.
Some of the refactoring to move I/O from main.c was done with in-place editing in mind, and is necessary to implement sed-style in-place editing. But it's not a high priority for me at this time, meaning: send us a PR if you want this done soon :)
@svnpenn - My apologies if I over-generalized, but the GNU awk documentation characterizes "inplace editing" as "an extension" (https://www.gnu.org/software/gawk/manual/html_node/Extension-Sample-Inplace.html). Also, your example does not work with the GNU awk installed on my machine (4.0.2), and the awk that comes with OS X 10.9.5 does not support a "-i" option at all.
I agree that jq can and should do in-place. The earlier problems were due to a poor original implementation.
The original implementation was poor and, in petticoat, had bad semantics.
I refactored the way I/O is done in main.c in order to make it easier to
later add sed-like -i -- it should be easy enough now. Send a PR or be
patient :)
Is this feature in the next release? I found it in 'release candidate 1', or is that the version that was retracted?
@danielbayley No, it got taken out. 1.5rc1 implemented the wrong -i semantics., sorry.
Until there is a "-i" builtin option, could you add something like
tfn="$(mktemp file.XXXXXX)" && jq '.' file >"$tfn" && mv "$tfn" file
to the Examples section of the manpage? And possibly add "sponge(1)" to the SEE ALSO section? When the inplace tool becomes more popular (included in moreutils?), it should be linked too.
@svnpenn you mean the ex(1)-Examples?. ex uses a tempfile internally. But anyway:
My suggestion is basicly to include anything appropiate in the examples to guide people in the right direction. Having seen the above http://backreference.org/2011/01/29/in-place-editing-of-files/ I was surpsied, on how many stupid ideas are out there to solve this problem. If that anything is the ex(1) example, also fine.
With the intention of "Hey user, you're looking for -i? Well it's not there yet for various good reasons, and well, you already have plenty of options to do this correctly with already existing tools".
There is now a Q&A about "in-place editing" in the FAQ: https://github.com/stedolan/jq/wiki/FAQ#general-questions
I think tee can be used as a workaround:
jq '.foo' bar.json | tee bar.json > /dev/null
(Output redirection to the black hole is optional if quiet mode is not strictly necessary.)
Ah, good call, @svnpenn!
If jq fails the whole bar.json will be truncated, so must be used with care ;-)
Some recommended and some not-recommended workarounds are covered on the FAQ page at "General Questions": https://github.com/stedolan/jq/wiki/FAQ#general-questions
The workaround with a temporary file can be hardened the following way for use in scripting:
jq '.foo' bar.json > baz.json && mv baz.json bar.json
Otherwise, it will also lead to an empty file in case of jq failure.
If this thread is not a proof the feature would save people a lot of time, I don't know what is.
@svnpenn - Neat. If that works on Windows, then it would be worthwhile having your slurp() packaged as sponge.bat since neither moreutils in general nor sponge in particular seems to be available via choco.
contrary too @mfilenko suggestion using tee is not a workaround: https://askubuntu.com/a/752451
I know, those are both referenced in the FAQ.
Just don't use tee (or direct > in general)
EDIT: just noticed there is a warning against tee in the FAQ too
Here is a more pragmatic question: What alternatives are for in-place editing of yaml or json files?
I am asking this because based on the history of this bug one thing is clear: the author doesn't want this in and it will not happen. If it didn't happen in 4 years, I doubt it will happen too soon so it makes sense to look for alternatives.
I am saying this because the last of this feature breaks the usability completely because it makes almost impossible to write reliable and easy to maintain scripts without it.
@pkoppstein OS X also has -i on awk, is just that its implementation is a little bit different than GNU awk, but is still possible to write code that works for both blends.
@ssbarnea You can use sponge from moreutils together with jq:
jq . your.json | sponge your.json
For me the lack of this feature was a reason enough for not using jq at all. I need to change a config file as part of our build process and this involves changing it multiple times during the process. Mainly we successfully used crudini in the past for doing the same to ini files.
The proposed workaround of using wrappers or other extra tools takes the dependency chain too far and we all know that we need and want to avoid having a kitchen-sink dependency chain. At the same time we want to keep out shell scripts clean and easy to read and update.
I think the options have been spelled out pretty clearly here. Either:
- Send a PR
- Add a :+1: to the description above and wait
- Fork this repo and do whatever you like
It's not the author's responsibility to implement this feature. I've added my vote above as I think this would be a useful feature, but there's no amount of elapsed time that will make this the responsibility of the maintainers to implement. Remember, they have already contributed untold hours of work to this project, to your benefit, for free, to provide all the other features you find so useful.
I'm not trying to cause offence here, just let's remember that we should be grateful for the time that others have spent on this project already, not demanding more. If you need something, implement it and please (if you are so inclined) share your implementation with the rest of us.
In case this helps someone, if/until this is implemented - here's another shortcut to achieve this using bash but without temporary files and other tools (bash & jq only):
TJQ=$(cat ./my.json | jq ".myProp = \"newValue\"") && echo ${TJQ} > ./my.json
Sorry if this has already been suggested - couldn't find it in the comments above, but I might have missed it.
I don't like to put the full output in memory, but in this case the code can be clarified a bit. Subtle changes, not tested:
TJQ=$(jq '.myProp = "newValue"' < ./my.json)
[[ $? == 0 ]] && echo "${TJQ}" >| ./my.json
The main changes are disable the field spliting in the echo argument and
remove one cat subprocess.
+1 :( Once again I want to reiterate that this as a native feature of JQ would be very useful. I frequently finding myself needing to merge two or more json data sets. This means there is going to be a source and a destination.
If the source data set has 1000 objects and the destination data set has 10,000 objects we will need to maintain the extraneous objects that are in the destination data set. Unless there is something I am missing I don't actually see a way to do this without producing a new object and thus a new file.
Creating a tmp file and moving it back to the original does work but seems like a sad workaround
Bump
I vote against this. jq should remain a stream editor, and not have to deal with the filesystem other than opening files for reading.
There are many edge cases to think about.
- A temporary file must be created in the same directory (to be renamed over the original at the end). How much effort should be made to make sure it's a unique name that won't accidentally overwrite an existing file?
- How much effort should jq do in copying the metadata? should it copy ownership? it can only do that if it's running as root. Should it copy permission modes or just rely on umask? should it try to detect if the original file had any OS-specific extended attributes and attempt to duplicate those? what about ACLs?
- What if the file is a symlink? should it follow the symlink and edit the regular file it targets, or replace the symlink with a regular file, effectively breaking the link? What if it's not a regular file? abort, or replace it with a regular file?
Whatever choices it would end up taking from the above, would be the correct ones in some context, but not in others. In my opinion it's better to let a third-party deal with these choices rather than burdening jq with them.
What if the distribution would come with a separate executable jq-inplace? This could be a shell script.
The reason I want in-place editing is because I want to mass update many files. jq already lets me do that, but > will only write back to a single file. Correct me if I'm wrong, but sponge, inplace and the mv hack only work on one file?
I can probably cobble together some kind of for-loop with my pathetic bash skillz but it'd be nice if someone posted an elegant solution for that.
I want to mass update many files.
Since you have a particular scenario in mind, you would probably be best off writing a script to do what you want, maybe using sponge (moreutils) or inplace (https://github.com/nicowilliams/inplace). The general problem is quite hard to state, let alone solve, because there are so many ways to invoke jq with multiple input sources. E.g. consider: jq . <(command) file1.json file2.json.
@mnpenner With a for-loop you'd have your jq script write to a new file, then move it over the original file if jq succeeded:
for file in ./*.json; do
jq '.[]|the|jq|script' "$file" > "$file.tmp" && mv "$file.tmp" "$file"
done
This is essentially what GNU's sed -i does (and what jq -i would do if it were implemented), it just does it transparently, and also tries to copy some of the file's metadata.
I worked out another solution:
find /some/dir -name "*.json" -exec inplace {} jq 'filter_here' \;
I couldn't get sponge to work because find/-exec doesn't like the pipe. Loop has fewer dependencies though, so thanks for that @geirha
@mnpenner - To use sponge with find, you could write s.t. like:
find /some/dir -name "*.json" | while read -r f ; do jq 'filter_here' "$f" | sponge "$f" ; done
@pkoppstein It should be
find /some/dir -name '*.json' -print0 | while read -r -d '' f ; do jq 'filter_here' "$f" | sponge "$f" ; done
because file names can contain newlines.
Wouldn't
jq -i 'filter_here' *.json
be much better? And more safe to use?
Isn't this the world we all want to live in? Power editing of json files right at our fingertips?
@metashock - The proposed -i option would probably not work that way (*), which is in a sense the main point - previous attempts to implement the feature have run into one problem or another. (One is tempted to say that the feature sounds great until confronted with someone else’s implementation of it.) See the most recent posts by joelittlejohn and geirha above.
As for my suggestion, please note that I did say “s.t. like”, precisely because of the “edge case” you mention.
(*) And if it did, then you’d still need sponge in other cases.
I reject to add -i to JQ. Is a bad idea.
This is a very old theme, solved by Pike and Kernigan since 1984, in section 5.5 (Replacing a file: overwrite), page 155 in the book:
https://cs.uwec.edu/~buipj/teaching/cs.388.s14/static/pdf/upe.pdf
This book is a MUST!
Also a must reading is UNIX Style, or cat -v Considered Harmful:
http://harmful.cat-v.org/cat-v/
Joan;
do you know any unixes that have the "overwrite" command that you have so kindly referred us to in this 30 year old work?
Furthermore, even use of "overwrite" (or sponge or inplace) would have to work in conjunction with the various find/for loops that users are having trouble constructing safely and reliably.
It's no answer.
Generally:
jq is a stream editor, perl is (among other things) a stream editor, awk is a stream editor, sed is a stream editor (THE Stream EDitor)
jq is the only one without -i, though GNU awk was late, getting -i just over 5 years ago.
Consider that sed -i is supported in BSDish and GNUish implementations, but not part of POSIX or Single Unix Specification. Users will have their -i
If you have a [files...] section (and jq does) then this is where -i applies. Each time the file is opened, refresh stdout.
Of course there may be invocations where -i won't make sense, Unix is full of such contradictions.
Be assured, jq will get -i, the only question is how much gnashing of teeth and user time and expense will be wasted playing code golf and fighting over it before it happens.
Sam
On 6 August 2018 at 10:59, Joan Josep Ordinas Rosa <[email protected]
wrote:
I reject to add -i to JQ. Is a bad idea.
This is a very old theme, solved by Pike and Kernigan since 1984, in section 5.5 (Replacing a file: overwrite), page 155 in the book:
https://cs.uwec.edu/~buipj/teaching/cs.388.s14/static/pdf/upe.pdf
This book is a MUST!
Also a must reading is UNIX Style, or cat -v Considered Harmful:
http://harmful.cat-v.org/cat-v/
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-410655342, or mute the thread https://github.com/notifications/unsubscribe-auth/AFf3zHq9iTRn4Bkv1WOwKyyiShfKti_gks5uOBOdgaJpZM4AmJxc .
herestrings use temp files
cat <<< "$(jq ".stuff += [ \"thatsthestuff\" ]" < config.json)" > config.json
also jj does have -i -o flags to pass files
herestrings use temp files
cat <<< "$(jq ".stuff += [ \"thatsthestuff\" ]" < config.json)" > config.jsonalso
jjdoes have-i-oflags to pass files
this is not portable and it works on bash and maybe a few others shells.
herestrings use temp files
cat <<< "$(jq ".stuff += [ \"thatsthestuff\" ]" < config.json)" > config.jsonalso
jjdoes have-i-oflags to pass filesthis is not portable and it works on bash and maybe a few others shells.
In addition, if you make a typo in the jq filter, or jq otherwise fails, config.json will be truncated to 0 bytes.
jq . your.json | sponge your.json
Today, I had the use-case of having to format multiple json files in a folder while also taking into account subfolders.
I wrote a small script to achieve that:
#!/bin/bash
set -euxo pipefail
shopt -s globstar
for file in **/*.json; do
jq . $file | sponge $file
done
(This script has to be in the folder where your json files are located. I may update this one day with the path being parameter.)
jqi() {
cat <<< "$(jq "$1" < "$2")" > "$2"
}
jqi ".stuff += [ \"thatsthestuff\" ]" config.json
There's value to adding a feature for usability. '-i' is well understood, and would make a lot of sense here.
This is a very old theme, solved by Pike and Kernighan since 1984, in section 5.5 (Replacing a file:
overwrite), page 155 in the book:https://cs.uwec.edu/~buipj/teaching/cs.388.s14/static/pdf/upe.pdf
This book is a MUST!
100% agree!! But none talks about unix design philosophy or intra process operability anymore so we're stuck with anti-unix practices like '-i' flags or all in one tooling that can't talk to each other.
For those that want to read this amazing document. The link above is missing but has been uploaded by the archiveteam: https://web.archive.org/web/20170720132901/https://cs.uwec.edu/~buipj/teaching/cs.388.s14/static/pdf/upe.pdf
Probably because -i works nicely for multiple files. Thankfully there has been progress since the AT&T days.
Sam
On Fri, 11 Jun 2021, 18:09 Dwight Spencer, @.***> wrote:
This is a very old theme, solved by Pike and Kernighan since 1984, in section 5.5 (Replacing a file: overwrite), page 155 in the book:
https://cs.uwec.edu/~buipj/teaching/cs.388.s14/static/pdf/upe.pdf
This book is a MUST!
100% agree!! But none talks about that anymore so we're stuck with anti-unix practices like '-i' flags. For those that want to read this amazing document. The link above is missing but has been uploaded by the archiveteam:
https://web.archive.org/web/20170720132901/https://cs.uwec.edu/~buipj/teaching/cs.388.s14/static/pdf/upe.pdf
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-859722954, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL7PTHWHXGAT73FNZVEKPDTSI7OBANCNFSM4AEYTROA .
because -i works nicely for multiple files
meh.. that's what tee(1) is for and that only works if your iterating over files which is again anti-practice;
for x in foo fu bar; do something $x; done Is a lot slower than xargs -I{} -P3 something {} <<< foo fu fubar or if one passes many '-i filename(s)' flags. In both cases they make for really unreadable scripting and a lot of codegolf'd one offs. Which is why I stand on the grounds of anti-practice.
My debate really isn't about should jq have an inplace flag, that might benefit many. But more about good scripting in general.
Most commands should be seen equally to a library method. jq just happens to give us the missing jsonpath method which other interpreter languages have baked in. In most programming best practices the fewer arguments you give to the method/function the better. Anything longer should be an interpreted file for repeatability (see awk -f for an example). Need that method/function to do more, write a wrapper to it.
It's the modern authoritarian way, justifying itself with the days when there was no choice.
"Must stop people using features that I don't know how to use"
And so missing features must be replaced by idiomatic expressions everywhere they are needed, and are even harder to get right.
Those who don't know how to use -i don't need to use it, but they seemed so determined to prevent anybody else succeeding with it.
Do they really look at people using sed -i and not know what is going on, or think that they would be just as easily able to parse an idiomatic loop and prove that it did the same thing? Surely these are the kind of people who write interview questions.
-
Prove that this expression is functionally the same as sed -i on the list of files. (Time allowed 10 minutes)
-
Explain why it is superior to sed -i (Time allowed 5 minutes)
-
Explain how it is more obvious that the expression replaces the files in place than it is that -i replaces the files in place (Time allowed 45 minutes)
Bonus points of the interviewee can come up with their own expression which only fails on unlikely corner cases, or "rare" race conditions
On Mon, 9 Aug 2021, 22:27 Abhiroj Panwar, @.***> wrote:
I am honestly amazed. 😲
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-895562165, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL7PTCLWIFX6ORL2KSJXQTT4BB4PANCNFSM4AEYTROA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .
Although I use below, still want '-i' or '-w'
jq 'del(.packageManager)' package.json | sponge package.json
It can be done with:
echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' /etc/docker/daemon.json)" > /etc/docker/daemon.json
Don't even need the echo.
On Wed, Oct 20, 2021, 18:02 Iaroslav Pleten @.***> wrote:
It can be done with:
echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' /etc/docker/daemon.json)" > /etc/docker/daemon.json
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-948099357, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUILSHGDJM7AUS5KQZ34FTTUH5DAJANCNFSM4AEYTROA .
Don't even need the echo. … On Wed, Oct 20, 2021, 18:02 Iaroslav Pleten @.***> wrote: It can be done with: echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' /etc/docker/daemon.json)" > /etc/docker/daemon.json — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#105 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUILSHGDJM7AUS5KQZ34FTTUH5DAJANCNFSM4AEYTROA .
@omgdenzuko Yes, you need, unless you are not using bash/zsh/...
Ok.. Since when did any posix shell not support proper stdout redirecting in the past 40-ought years?
No. Echoing a subshell's output as a string is, I repeat on K&R's graves, not needed. Its like piping into cat from cat.
jq '
On Wed, Oct 20, 2021, 18:54 loynoir @.***> wrote:
Don't even need the echo. … <#m_-1820777542152595716_> On Wed, Oct 20, 2021, 18:02 Iaroslav Pleten @.***> wrote: It can be done with: echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' /etc/docker/daemon.json)" > /etc/docker/daemon.json — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#105 (comment) https://github.com/stedolan/jq/issues/105#issuecomment-948099357>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUILSHGDJM7AUS5KQZ34FTTUH5DAJANCNFSM4AEYTROA .
@omgdenzuko https://github.com/omgdenzuko Yes, you need, unless you are not using bash/zsh/...
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-948119862, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUILSHENEPYRKJFKS5YGTK3UH5JDNANCNFSM4AEYTROA .
@omgdenzuko
Don't even need the echo.
$ echo '{"a":"b"}' > daemon.json && jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json > daemon.json && cat daemon.json
$ echo '{"a":"b"}' > daemon.json && echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json)" > daemon.json && cat daemon.json
{
"a": "b",
"registry-mirrors": [
"https://docker-mirror"
]
}
@omgdenzuko I'm not saying I'm disagreed.
jq '<expression>' > output.jsonworks on all posix systems.
But not agree with your
Don't even need the echo.
Because jq '<expression>' inplace.json > inplace.json truncate inplace,
while @ipleten using echo "$(jq '<expression>' inplace.json)" > inplace.json to edit files in place
Yeah that's the usual *sh not holding a file buffer after a return from subshell. It's exactly why tee(1) exists.
Just pipe stdout to tee or read so one gets stdout as a memory block to flush to disk. That's what your echo is doing while adding complexity, it's capturing stdout from the subshell then parsing it as a string, looking for shell expansions to evaluate then dumping it to its stdout where the shell then redirects it to a file.
jq &1 pipe tee file eliminates the evaluation and just writes to the file.
On Wed, Oct 20, 2021, 22:33 loynoir @.***> wrote:
@omgdenzuko https://github.com/omgdenzuko
Don't even need the echo.
$ echo '{"a":"b"}' > daemon.json && jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json > daemon.json && cat daemon.json
$ echo '{"a":"b"}' > daemon.json && echo "$(jq '. + {"registry-mirrors": ["https://docker-mirror"]}' daemon.json)" > daemon.json && cat daemon.json
{ "a": "b", "registry-mirrors": [ "https://docker-mirror" ] }
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-948228135, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUILSHHMYVGEGUUUQVFNLD3UH6CYLANCNFSM4AEYTROA .
@omgdenzuko Oh, seems I misunderstood your commment
Don't even need the echo.
On ipleten's approach
echo "$(jq '<expression>' inplace.json)" > inplace.json
To
jq '<expression>' inplace.json > inplace.json
But seems you actaully means
jq '<expression>' inplace.json | tee inplace.json
I prefer sponge or echo approach then tee approach
https://stackoverflow.com/questions/33638511/differences-between-sponge-and-tee
Using > or tee is at best a bad idea unless your shell or version of tee specifically documents that it can be used in the desired way.
For more on this, see http://askubuntu.com/questions/752174
Using sponge is often the way to go.
Hey!
I'm glad I triggered such discussion and really glad so many people know *nix shell. sponge isn't included by default in almost all docker containers(frankly I've never heard about it before).
I just put one of many possible solutions.
Thanks!
@ipleten in any debian based distro sponge is apart of the moreutils and its not part of POSIX or heirloom *nix so that's why its not include by default.
A list of supported package managers to install sponge and it copy-pasta command can be found at https://command-not-found.com/sponge
This repeated discussion, the usual mistakes, the file races, repeated all over the world (not only here) is why in-place option is needed and why it is futile to argue about how the unix philosophy renders it not strictly necessary.
It is needed, because too often people can't get the workarounds right.
The sheer number of hours wasted must be incredible and only get worse as jq becomes more popular in spite of this obvious deficiency.
Sam
On Thu, 21 Oct 2021, 20:42 Dwight Spencer, @.***> wrote:
@ipleten https://github.com/ipleten in any debian based distro sponge is apart of the moreutils and its not part of POSIX or heirloom *nix so that's why its not include by default.
https://command-not-found.com/sponge
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stedolan/jq/issues/105#issuecomment-948946635, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL7PTBNP4CXZ3O26VAQK7DUIBUJVANCNFSM4AEYTROA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
yq does support inplace editing and can produce normal JSON. CLI API is different however.
$ echo '["first"]' > message.json
$ yq -i -ojson eval '. += ["second"]' message.json
$ cat message.json
[
"first",
"second"
]
https://mikefarah.gitbook.io/yq/
I use yq,
For advanced users this -i (in-place update) is needed, hope be added to jq
yq -iP '.Email.Port=3030' config.json -o json
-iin place update-Ppretty print-ooutput should be json
After nearly 10 years of jq maintainers arguing against in-place editing on the grounds that there are difficult and obscure ways around that delightful limitation, we now also have jaq, written in rust, with in-place editing of multiple files, and fixing various jq bugs:
https://github.com/01mf02/jaq#readme