ERROR: Cannot set preferences of an unknown package that is not loaded!

[schlichtanders]

Trying to define the preferences for RCall, I am running into an error. How to actually do this?

julia> import Pkg; Pkg.add(["UUIDs", "Preferences", "CondaPkg"])

julia> using UUIDs, Preferences, CondaPkg

julia> envdir = CondaPkg.envdir()

julia> RCALL_UUID = UUID("6f49c342-dc21-5d91-9882-a32aef131414")
UUID("6f49c342-dc21-5d91-9882-a32aef131414")

julia> set_preferences!(RCALL_UUID, "Rhome" => "$envdir/lib/R", "libR" => "$envdir/lib/R/lib/libR.so")
ERROR: Cannot set preferences of an unknown package that is not loaded!
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] set_preferences!(::UUID, ::Pair{String, String}, ::Vararg{Pair{String, String}}; export_prefs::Bool, active_project_only::Bool, kwargs::@Kwargs{})
   @ Preferences ~/.julia/packages/Preferences/enLcN/src/Preferences.jl:281
 [3] set_preferences!(::UUID, ::Pair{String, String}, ::Pair{String, String})
   @ Preferences ~/.julia/packages/Preferences/enLcN/src/Preferences.jl:223
 [4] top-level scope
   @ REPL[8]:1

[schlichtanders]

It seems the only way to actually fix this is to first add the RCall package (which will trigger precompilation of RCall) and then add the preferences like above.

I tried writing the LocalPreferences.toml directly, but it won't actually work, has_preferences will simply show false as long as RCall is not a top-level dependency, even if the preferences are set inside LocalPreferences.toml

[palday]

I think this setup is also desirable behavior -- I think it makes sense to require direct dependency for something you care enough about the details of to set a preference on.

[schlichtanders]

The confusing part is on reading the preferences. Despite having LocalPreferences.toml filled with an entry for RCall, you will get the following divergent behaviour between UUID and package-name, if RCall is not added as a toplevel dependency.

julia> using Preferences, UUIDs

julia> RCALL_UUID = UUID("6f49c342-dc21-5d91-9882-a32aef131414")

julia> has_preference(RCALL_UUID, "Rhome")
false

julia> has_preference("RCall", "Rhome")
ERROR: ArgumentError: Cannot resolve package 'RCall' in load path; have you added the package as a top-level dependency?
Stacktrace:
 [1] package_lookup_error(name::String)
   @ Preferences ~/.julia/packages/Preferences/enLcN/src/utils.jl:88
 [2] has_preference(name::String, key::String)
   @ Preferences ~/.julia/packages/Preferences/enLcN/src/Preferences.jl:76
 [3] top-level scope
   @ REPL[8]:1

[frankier]

This might be relevant/helpful: https://github.com/JuliaPackaging/Preferences.jl/issues/53

[frankier]

Okay so we have the following ways to set preferences that we could advise people of in the documentation:

1. Manually edit LocalPrefererences.toml file
2. Use the Preferences module
3. Use the PreferenceTools module

1. has the disadvantage that it can result in preferences that don't take for transitive dependencies that don't

2. won't work unless RCall is already in the Project.toml, but 2a) since Pkg 1.11, we can use Pkg.add("RCall"; extra=true) beforehand and 2b) in the next release of Preferences we can set the preference as long as we supply RCall.jl's UUID too. This will ensure RCall is added to the extras only if need be.

3. will work today, but the user has to installed PreferenceTools too. This is what CondaPkg does: https://github.com/JuliaPy/CondaPkg.jl?tab=readme-ov-file#preferences

I think standardising on 2b in RCall's documentation guiding the usage of Preferences, both with and without CondaPkg would be best. We just need to wait until the next release of Preferences.jl.