kscript
kscript copied to clipboard
kscripting
Support local jar references in @file:DependsOn
I'm struggling to understand how I'm supposed to get access to my local libraries in kscript. I have simple file :
#! /usr/bin/env kscript
import se.explodera.util.sql.OracleDBFileLoader
println("Hello.")
where the imported class can be found in a jar file on the environment classpath. But when I run the script I get the message
error: unresolved reference: se
import se.explodera.util.sql.OracleDBFileLoader
Do I really have to add my local jar file to a maven repository?
When I try to print the classpath from within my code it seems to be limited to kotlin-runner.jar. Is this by design, or am I doing something wrong?
Either you wrap them in a jar and declare them as a dependency with @file:DependsOn (see https://github.com/holgerbrandl/kscript#declare-dependencies-with-deps)
Or you could simply include the OracleDbFileLoader with @file:Include (see https://github.com/holgerbrandl/kscript#ease-prototyping-with-include)
And what is the syntax?I tried @file:DependsOn("/absolute/path/to/the/jar/file")?, without success.
gradle publishToMavenLocal or mvn install @file:DependsOn("vendor:name:version"")
You can not reference a jar directly
Thanks, @holgerbrandl. I gave up on trying to reference a local file directly two weeks ago, and added the jar to my local Maven repository (file cache) instead. It's a nuisance to have to learn details about Maven for a simple thing like referencing a local jar, but then again, it works well once you have done it.
If anybody else reads this thread and wonders how I did that, I wrote up my experiences in a post on my web page.
Indeed it's suboptimal from a user experience perspective. I've changed the title to reflect the idea better. I guess it should be addressed eventually by either extending the existing DependsOn annotation or by adding a new one. PRs are welcome.
I am not really convinced that referring to local files from annotation DependsOn is the right way of solving the need to use local jars. First of all, I don't think it is the standard in any way and JetBrains most probably won't support it. And I think that I know why: referring to local files from remote scripts (as kscript can load scripts from URLs) will be always a security risk. Additionally, it will encourage writing not reusable scripts without really good reason. I think that solution with publishing jars to local Maven/Gradle repositories is the best way of handling the issue.
Alternatively, maybe we can define the env variable KSCRIPT_LOCAL_JAR_DIRECTORY and then, whenever the maven jar is not resolved, try to resolve the jar from that directory:
eg. DependsOn("net.myorg:myjar:1.3") --> file: net.myorg_myjar_1.3
JetBrains most probably won't support it ... will be always a security risk.
If that's the case, it's quite sad. That means you have to jump through a few hoops you shouldn't have to. Wouldn't it make more sense to prevent a script fetched remotely from having references to local jars, rather than all scripts? The idea that a completely private, local script can't be simply written to refer to a completely private, local jar ... seems like it's making simple things hard.
I agree @aartiPl, the with comes up once in a while, but I also don't consider it a clean solution. It lacks portability as you explained very well.
I also don't really like the concept of KSCRIPT_LOCAL_JAR_DIRECTORY
Should this issue be closed as wontfix, then? To save people wasting time on it?
By the way, why is running remote scripts with references to local jars more of a security risk than running remote scripts without them? I couldn't see the security angle. I can certainly see that scripts with local jar references are brittle, and I would only use them for private local scripts (e.g. quick, exploratory local testing of features in libraries which I'm developing, before I get to a point where I can write proper tests for the feature) - I see that as a genuine use case.
I agree security is not addressed here at all. If the user decides to run whatever script, its his or her responsibility to make sure that code won't do any harm.
My concern was purely about portability: The main intent of kscript was/is to enable self-contained mini-applications. Having a local file reference violates that principle.
So indeed won't fix. Sorry for the inconvenience.
So indeed won't fix. Sorry for the inconvenience.
No problem. Note that I use this functionality with --package to generate completely self-contained mini-applications, and the local jars get combined into the final output anyway - the local reference is only temporary, and not present in the final result. Works for me!
@holgerbrandl , @vsajip - after rethinking KSCRIPT_LOCAL_JAR_DIRECTORY concept presented by me above, I also agree that it won't be the best way to go. But please let me propose another variation on that:
-
We will have env variable KSCRIPT_LOCAL_JARS, which will contain a set of paths to jar files, which should be included in -classpath of the script. That should already help with the presented use-case (quick scripting for personal usage): setting up such an env variable will make functionality available in script. No changes to the DependsOn annotation is needed. Also implementation should be rather simple.
-
In the long term, we should develop a configuration file for kscript (in appDir - infrastructure is already there), which can keep such configuration details, so that it is not necessary to provide env variables every time.
If you think that this use case is important enough, I think we can reopen the ticket and maybe someone will provide implementation.
@aartiPl Please correct me if I'm wrong, but are you saying that in this line (and the equivalent in kscript.bat),
https://github.com/holgerbrandl/kscript/blob/dc827753adf707cf31cd220716e85139cae2d9cd/src/kscript#L49
the JAR_PATH should take into account the value of KSCRIPT_LOCAL_JARS, if defined? Would that be enough for import statements in a .kts file to work correctly? If so, it sounds promising.
Nope. The flow should be like that:
export KSCRIPT_LOCAL_JARS=/home/myjars/myjar1.jar;/home/myjars/myjar2.jar
Then in ConfigBuilder, if the env variable is set, it should be parsed, changed to the list of OsPath, and assigned to localJars in ScritiptinConfig in Config.
Then it should be plainly added to the resolvedDependencies set.
And basically, that should be all. Quite a simple implementation without the need of changing parser and DependsOn syntax. If you want to provide PR, please reopen ticket.
I don't agree closing this issue here. "Self-contained mini-applications" doesn't mean only single file is allowed. A local jar, with a path relative to the kts file, should also be a valid part of a self-contained mini-applications.
Many production machines prohibit the downloading of jar files in maven, and so local jar file is the only way to go. Why not make this easier, without forcing us to do it in the hard(er) way of setting up a local maven repository?
Sorry I missed the KSCRIPT_LOCAL_JARS environment variables part. This should make things slightly easier.
Ok, I will reopen the ticket then. After I make some additional refactoring on kscript, I will look at the implementation.
Thank you. In addition, it'd be nice if we can support importing a whole local, relative directory of jars, with option to specify whether it's recursive.
The jar loading order should be based on alphabetical ordering of their subfolder and file names. This can at least provide a consistent behaviour, if we don't have a better rule.
I can study and submit PR after you have done your additional refactoring, if you want.
@sken130 - there is an initial implementation in the 4.2 branch (Please look at the initial @vsajip implementation, which was used as a base for my changes: https://github.com/vsajip/kscript/commit/0118a9b429c39c97c49fa36460b2fc4204149eaa - the same files are touched). Walking through directories should always be the same, so we don't need sorting. I am only missing some tests. Any help here would be warmly welcome.
Thanks, give me a few days, I have just forked the repo and set up the environment in Windows.
To add a test, just create a ****Test.kt file under src/test/kotlin/io/github/kscripting/kscript?
Great that you want to take on this task - I appreciate that!
src/test/kotlin/io/github/kscripting/kscript - here you can find unit tests. maybe e.g. recursive reading of jar/aar files can be tested here. /integration/kotlin/io/github/kscripting/kscript/integration - all the integration tests have its home in this directory; most probably those tests will be more important to fully test the feature.
Just prepare a first version of PR - I will have a look at it, and advice how to proceed if needed.
KScript 4.2.0 allows referencing local jar files in a given directory. You can provide this directory by putting its path into KSCRIPT_DIRECTORY_ARTIFACTS environment variable or through the configuration file and its property: scripting.directory.artifacts.
Future versions of kscript may also handle local file references in DependsOn, but it will solely depend on the Kotlin scripting standard. If it is allowed there, then it will also work in kscript. The option with the local jar files directory should stay anyway.
I am so sorry I still don't have time to work on the tests, but in the following week, it will be public holidays in my country, so there's 60% chance I will have time on this (write the tests if it's not written yet, or check/add a bit more tests if it's written already)
@sken130 - That's okay. I have already added some tests. But still, you can look and think about how to improve coverage. It can even be for different functionality, e.g., for Windows interoperability.