tjake
jlama-native shared library not found
hi i am trying to run the tjake model using langchain4j locally. the models able to generate responses but taking too much time generating responses. while running in spring i got the issuse with native library is not loaded.
""" c.g.t.j.t.operations.util.JarSupport : jlama-native shared library not found .g.t.j.t.o.TensorOperationsProvider : Native operations not available. Consider adding 'com.github.tjake:jlama-native' to the classpath c.g.t.j.t.o.TensorOperationsProvider : Using Panama Vector Operations (OffHeap)
""""
"""this is the pom.xml that i am using
4.0.0 org.springframework.boot spring-boot-starter-parent 3.5.3 com tablesaw 0.0.1-SNAPSHOT excel Demo project for Spring Boot 21 org.springframework.boot spring-boot-starter-web
org.projectlombok lombok true
org.springframework.boot spring-boot-starter-test test
''''
can you please check this and give me suggestions for this.
Can you run this with debug logging? it should show more details as to why it failed to load.
2025-07-28T10:33:31.033+05:30 INFO 17160 --- [excel] [ main] com.tablesaw.ExcelApplication : Starting ExcelApplication using Java 21.0.2 with PID 17160 (D:\tablesaw\target\classes started by Jaswanth.Manduri in D:\tablesaw) 2025-07-28T10:33:31.035+05:30 INFO 17160 --- [excel] [ main] com.tablesaw.ExcelApplication : No active profile set, falling back to 1 default profile: "default" 2025-07-28T10:33:31.794+05:30 INFO 17160 --- [excel] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port 1608 (http) 2025-07-28T10:33:31.805+05:30 INFO 17160 --- [excel] [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat] 2025-07-28T10:33:31.805+05:30 INFO 17160 --- [excel] [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.42] 2025-07-28T10:33:31.856+05:30 INFO 17160 --- [excel] [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext 2025-07-28T10:33:31.857+05:30 INFO 17160 --- [excel] [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 787 ms 2025-07-28T10:33:32.139+05:30 INFO 17160 --- [excel] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port 1608 (http) with context path '/' 2025-07-28T10:33:32.150+05:30 INFO 17160 --- [excel] [ main] com.tablesaw.ExcelApplication : Started ExcelApplication in 1.413 seconds (process running for 1.706) 2025-07-28T10:33:32.701+05:30 WARN 17160 --- [excel] [ main] c.g.t.j.t.operations.util.JarSupport : jlama-native shared library not found 2025-07-28T10:33:32.703+05:30 WARN 17160 --- [excel] [ main] c.g.t.j.t.o.TensorOperationsProvider : Native operations not available. Consider adding 'com.github.tjake:jlama-native' to the classpath 2025-07-28T10:33:32.719+05:30 INFO 17160 --- [excel] [ main] c.g.t.j.t.o.TensorOperationsProvider : Using Panama Vector Operations (OffHeap) 2025-07-28T10:33:32.731+05:30 INFO 17160 --- [excel] [ main] c.g.tjake.jlama.model.AbstractModel : Model type = Q4, Working memory type = F32, Quantized memory type = I8
Response of Model for Hello!: How can I help you today? Is there something specific you would like to know or discuss? I'm here to assist you with any questions or topics you have. Let me know if you need any help!
The model is loaded and it is running sir. I am getting the responses but my concern is that Jlama-native library is not found and it is using the default panama vector operations.
Is that an issue when native operations are not found.
And one thing to clarify sir
when i use prompt like this and send it to the model why it is taking too much time to generate response. For simple question it is generating the response fast like for above Hello it has generated it in 10 sec.
But for below prompt it is taking time like around 10 min to generate the response
ChatModel model = JlamaChatModel.builder()
.modelName("tjake/Mistral-7B-Instruct-v0.3-Jlama-Q4")
.temperature(0.1f)
.build();
String prompt = """
You are a Python pandas expert. Given the following Excel data information and user query, generate ONLY the pandas code to answer the question.
DATA INFORMATION: %s
USER QUERY: %s
IMPORTANT INSTRUCTIONS:
1. DataFrames are loaded in 'dfs' dictionary, keyed by file names (e.g., dfs['file1.xlsx'])
2. Generate ONLY executable pandas code
3. Use proper pandas methods for filtering, grouping, aggregation
4. Identify join keys from column names or sample data
5. End with assign result to 'result' variable
6. Do not include explanations, just the code
7. Handle potential missing values appropriately
PANDAS CODE:
```python
""".formatted(dataInfo.toString(), userQuery);
System.err.println("prompt: " + prompt);
String response = model.chat(prompt);
Sir can i get any suggestions on this
2025-07-28T10:33:32.703+05:30 WARN 17160 --- [excel] [ main] c.g.t.j.t.o.TensorOperationsProvider : Native operations not available. Consider adding 'com.github.tjake:jlama-native' to the classpath
If you are seeing this something is no right. The shared library will load fom the JAR.
I have a fork of Jlama here:
https://github.com/edwardcapriolo/deliverance/blob/323fe41c400e6f4b40257a5969d6e530674c595b/native/src/test/java/SimdTest.java#L40
When all else fails I call "load" directly in my code. I haven done specific benchmarking but based on your system even the panama tensors arent opimized for all systems. I was running on a neutenix XEN machine and the float with is 128 bits. A unit test ran for 12 hours. So do your best to get the native support working.
