julia
julia copied to clipboard
JuliaLang
Set temporary env whenever Julia is started with `--project=@temp`
- Update relevant docs.
- Add tests for
--project=--tempandJULIA_PROJECT="--temp".
This makes it convenient to start and run Julia in a temporary environment. This brings Pkg.activate(; temp=true) functionality to cmdline.
(temp edited to --temp)
Similar to https://github.com/JuliaLang/julia/pull/49061 (@KristofferC), and discussion there of whether we should spell this --temp to exactly match the Pkg REPL behavior
This could breaks existing scripts, though. How about using @temp (similar to how there is @.)
You have now changed it to --project=--temp. That's pretty unusual and fits into no other pattern, does it? Could you at least explain your rationale over the suggestion to use @temp, which would by the way also naturally fit in with the code in load_path_expand
The @abc pattern is already used to activate shared env abc in the envdir(). So, avoiding @temp might be better.
Although --project=--temp looks unusual, it follows the ] activate --temp pattern.
I really appreciate the functionality but sorry must it be spelled --project=--temp ? I can't think of a single other CLI app I've ever used with the pattern --arg1=--arg2. I understand that within Pkg it is ] activate --temp, but in this context it just seems extremely odd.
I much prefer the original --project=@temp (which as previously noted mirrors the new --project=@scriptdir) or something independent like --temp-project
can't think of a single other CLI app I've ever used
It can be a somewhat common pattern in compiler drivers like Julia, gcc, or clang. E.g. gcc -Wl,-z,undef or clang -mllvm
As conversations, here and in #49061, have gravitated toward --project=--temp, I've retained that format.
--temp-project looks very neat; but I'm not sure if a new argument that is a subset of another, would be a good idea.
reading through both these issue threads, I would not call it "gravitating," more just stalled after --temp was proposed. Just counting heads, I definitely see more contributors proposing or preferring other options to --temp (by my count of the thumb wars, it's 6-3). and the suggestion for --project=_ has 6x👍 from non-commenters
that being said, OSS is not a vote, and I am happy to be overruled. just wanted to put my feedback in that I find this syntax quite clunky.
@vtjnash that's a good point about the compiler drivers, but I strongly suspect that most users see Julia more like an app than they see it like a compiler
--temp is technically a legal directory name also though. I think the whole flags vs file heuristics that clis do is a bit of an antipattern (e.g. why there's often things like -- to separate them). I don't think it's any better than --project-temp though and I also think --project=@temp is fine (since @ is already special; yes I know that it currently means the temp shared environment - I don't think it's a big issue, but if people are concerned, it could only have the new meaning if the temp shared environment doesn't exist).
How about an independent command line option? You would invoke this as julia --temp. It is an error to provide both --project and --temp. I could go for --temp-project or --project-temp as well. Having the value of an option use a -- prefix looks strange to me and might even confuse some parsers.
--temp-project has the benefit that you could just type --temp because julia args only need to be complete enough to be unambiguous
in favor of --temp/--temp-project because @temp should be persistent just like any other @abc and --arg1=--arg2 is weird among all the other command-line interface.
--temp is technically a legal directory name also though
Just as an observation for everyone, the pattern to force it to be a directory is to prepend ./ to make it --project=./--temp if you really must use a relative path to a folder of that name. Same with @ being alternatively specified as --project=@./temp or --project=./@temp to force alternative meanings
Personally, I would prefer if this flag starts with --project so that it is easy to club it with the other forms in which --project= is used. From this perspective, I would prefer --project=@temp as it's the most visually similar form, but --project-temp also looks fine. --temp-project departs from this pattern, and even though it's easier to read than --project-temp, I would prefer consistency. The shortened form julia --temp looks cryptic to me, as it's unclear that the temp refers to the project.
Whatever is chosen here, please make it easy and convenient to use. Being able to write julia --temp is by far the best option I've seen suggested in this respect.
(As a side note, on international keyboards @ tends to be tucked away, e.g. as AltGr+2 on my keyboard.)
@tempshould be persistent just like any other@abc
persistent-ly empty, so it kind of is
something is going to have to be special-cased, and I think it's least surprising if that's behind an @ symbol --- the universal Julian mark of "non-standard stuff be here"
and I know it has been mentioned a few times so this is a redundant observation, but nonetheless I think it's going unappreciated that --project=@scriptdir has been already chosen to special case an @keyword project to be something other than a persistent shared environment
--project=@scriptdirhas been already chosen to special case an@keywordproject to be something other than a persistent shared environment
It was chosen but not really by consensus; there was no discussion in that PR about the clash before it was merged, and I'm not sure that issue was even known (I brought it up post-merge). It seems like a way to do it but it doesn't really seem clear that it's the best way, and it also doesn't seem totally to late to do something better, as it hasn't been released yet.
that's fair, but I just mean the two args should not live by different rules (in fact, it seems cleaner to me if scriptdir and temp are accessible in similar ways, since they both special-case the --project parameter)
so if --project-temp is the way to go here then --project-scriptdir should be so for the other PR
persistent-ly empty, so it kind of is
That's not how persistent is usually defined in computing realm I think.
https://en.m.wikipedia.org/wiki/Persistence_(computer_science) summarize it as:
refers to the characteristic of state of a system that outlives (persists more than) the process that created it.
So in this case the temp environment with non empty stuff is created by the Julia process, and thus is not persistent.
From an interactive user point of view, --project=temp and --project=scriptdir actually seems slightly better than --project=@temp or --project=@scriptdir, because shared environments are accessible globally, so those are always meaningful names, whereas --project=temp is only a clash when there's actually temp (or scriptdir) directory in the current working directory. (And having a shared environment named either does seem like a reasonable thing to have).
But from a tooling point of view, --project=temp seems worse because there is probably a lot of tools/CI scripts/etc that expect --project=$relpath to work for some large set of relpaths (that might not include ones starting with @, but probably does include ones like temp or scriptdir).
To me the idea of new CLI arguments like --project-temp and --project-scriptdir makes sense in that there's no clash at all, but then one needs to thread through a new CLI argument for each...
What about something like @temp but using new syntax to not clash as much (besides with unusually named local dirs, which was already a problem with the introduction of @ for shared envs and seems to have worked fine in practice)?
e.g. --project=+temp or --project=!temp or something like that? (and similarly for scriptdir)
I think that expressing this as --project=@temp and allowing @temp in LOAD_PATH to create a temporary directory would be a good approach here. That is a feature that came up in https://github.com/JuliaLang/Pkg.jl/pull/2669#issuecomment-884502870; I posted a suggestion on Slack and then @IanButterworth mentioned it on GitHub (thank you, Ian): the idea is to have a LOAD_PATH of @:@temp:@v#.#:@stdlib for @oxinabox's preferred workflow where she wants to load a package but not into the currently active project, nor into a persistent named environment. If we let @temp mean that, then her preferred workflow would become much easier to implement (currently, you'd have to do it in startup.jl). I agree with @keno that the concern that @temp might already be a named environment isn't a big one. It's weird to have a persistent named environment called temp.
Oh, I also don't like the name @scriptdir much and totally missed the discussion of it. I would prefer @tecosaur's proposal of @script that behaves like @. looking for a project directory in the ancestors of the path to the currently running script file.
If we use @temp I suggest that we should check if a shared environment named temp exists. We can then either
- Issue a warning and then just use the existing shared directory (backwards compatibility).
- Issue an error.
I'm learning towards option 1. It would be similar to a deprecation warning.
Additionally, we could offer a helpful error message about referring to the shared environment using its absolute path.
Oh, I also don't like the name
@scriptdirmuch and totally missed the discussion of it. I would prefer @tecosaur's proposal of@scriptthat behaves like@.looking for a project directory in the ancestors of the path to the currently running script file.
@scriptdir is new in 1.11. If you feel strongly, this can still be changed.
With the update,
-
--project=@tempnow starts Julia with a temp environment. -
LOAD_PATHis now["@", "@temp", "@v#.#", "@stdlib"]. This also means theoiny/n/ogives a temporary env as an option (follows the order in LOAD_PATH).
However it does not check for a shared env temp yet.
👍 I propose that we give @temp only one special meaning and not allow any shared env named temp.
To fully ensure this, Pkg should treat @temp, @stdlib and @. differently.
Current behavior:
(@v1.11) pkg> activate @.
ERROR: not a valid name for a shared environment: .
(@v1.11) pkg> activate @temp
Activating new project at `~/.julia/environments/temp`
(@temp) pkg> activate @stdlib
Activating new project at `~/.julia/environments/stdlib`
(@stdlib) pkg> activate @
Activating new project at `~/.julia/environments`
(@environments) pkg>
To new behavior:
(@v1.11) pkg> activate @.
# activate the current project
# While `--temp` continues to do what it does,
(@v1.11) pkg> activate @temp
Activating new project at `/tmp/jl_oqUHmk`
(@temp) pkg> activate @stdlib
Activating project at `@stdlib/v1.11`
(@stdlib) pkg> activate @
# activate the active project
julia> Pkg.activate("temp"; shared = true) # (edited)
┌ Error: `temp` is an invalid name for a shared environment.
│ To create a temporary environment use `Pkg.activate(; temp = true)`
└ @ Main REPL[10]:2
🚀 If instead Julia should check for a shared-env-temp, I'll modify load_path_expand as the snippet below:
the alternate
function load_path_expand(env::AbstractString)::Union{String, Nothing}
# named environment?
if startswith(env, '@')
# `@.` in JULIA_LOAD_PATH is expanded early (at startup time)
# if you put a `@.` in LOAD_PATH manually, it's expanded late
env == "@" && return active_project(false)
env == "@." && return current_project()
env == "@stdlib" && return Sys.STDLIB
if startswith(env, "@scriptdir")
if @isdefined(PROGRAM_FILE)
dir = dirname(PROGRAM_FILE)
else
cmds = unsafe_load_commands(opts.commands)
if any((cmd, arg)->cmd_suppresses_program(cmd), cmds)
# Usage error. The user did not pass a script.
return nothing
end
dir = dirname(ARGS[1])
end
return abspath(replace(env, "@scriptdir" => dir))
end
env = replace(env, '#' => VERSION.major, count=1)
env = replace(env, '#' => VERSION.minor, count=1)
env = replace(env, '#' => VERSION.patch, count=1)
name = env[2:end]
temp = name == "temp" ? true : false
# look for named env in each depot
for depot in DEPOT_PATH
path = joinpath(depot, "environments", name)
isdir(path) || continue
for proj in project_names
file = abspath(path, proj)
if isfile_casesensitive(file)
temp && @warn """
Activiating the pre-existing shared environment named `temp` at
$file."""
return file
end
end
end
isempty(DEPOT_PATH) && return nothing
temp && (return mktempdir()) || (return abspath(DEPOT_PATH[1], "environments", name, project_names[end]))
end
# otherwise, it's a path
path = abspath(env)
if isdir(path)
# directory with a project file?
for proj in project_names
file = joinpath(path, proj)
isfile_casesensitive(file) && return file
end
end
# package dir or path to project file
return path
end
Please indicate with 👍 or 🚀.
julia> Pkg.activate(; shared = "temp") ┌ Error: `temp` is an invalid name for a shared environment. │ To create a temporary environment use `Pkg.activate(; temp = true)` └ @ Main REPL[10]:2
I thought the shared keyword only accepted a Bool?
Also, we really should check for an existing shared environment called temp.
Modulo @mkitti's comment about not changing the default LOAD_PATH, this seems great and triage approves. 💟