djl

djl copied to clipboard

Please add JPMS/module-info data to this project

2

## Description For better usage in a modularized application and to prevent hacky workarounds, it would be great to provide the correct module info declaration. Especially for the ServiceProviders. Refs...

Siedlerchr

Ensure Batchifier returns compatible arrays for hybrid engines like OnnxRuntime

1

Addresses issue raised from https://deepjavalibrary.slack.com/archives/C01AURG857U/p1724804411722599. Essentially batchifying / unbatchifying OrtNDArrays produce PtNDArrays, and this causes downstream issues, particularly in OrtSymbolBlock:114: ``` OrtSession.Result results = session.run(container); NDList ret = evaluateOutput(results); ret.attach(inputs.head().getManager());...

david-sitsky

## Description ``` image_uri = image_uris.retrieve( framework="djl-tensorrtllm", region=sess.boto_session.region_name, version="0.29.0" ) ``` ``` model = sagemaker.Model( image_uri=image_uri, role=role, # specify all environment variable configs in this map env={ "HF_MODEL_ID": "mistralai/Mixtral-8x7B-Instruct-v0.1", "TENSOR_PARALLEL_DEGREE":...

gsjoy8888

Sagemaker + LMI async -> Invalid Default Sagemaker Metric

1

## Description Following the guidance of LMI15 async, the vllm config is ``` engine=Python option.async_mode=true option.rolling_batch=disable option.entryPoint=djl_python.lmi_vllm.vllm_async_service option.tensor_parallel_degree=max ``` However, Sagemaker default metrics are invalid and couldn't show the real...

ValkyriaLenneth

I am on Windows 10. Some months ago, I found out that having (multiple) Cuda installed on your system creates conflicts that causes the cuda contained in djl files not...

ThiloteE

use the gte model generate embedding

2

Issue Description: i use the gte model generate embedding ，but get a error like the following  the gte model download from [https://www.modelscope.cn/iic/nlp_gte_sentence-embedding_chinese-large.git](url) my code: ` { public static void...

g5zhu5896

could straightly download the djl basic datasets by url

4

Hi, I want to use djl cleaned basic datasets for view, https://djl.ai/docs/dataset.html, but I only find write java djl code ,but I want to download straightly by url, could you...

mullerhai

## Description After a few times creating a DJL NDArray (~100 times), it segfaults. ### Expected Behavior Creating an NDArray doesn't segfault. ### Error Message ``` Stack: [0x00006b3f802fc000,0x00006b3f803fc000], sp=0x00006b3f803f8be0, free...

Martmists-GH

## Description currently the only termination criteria is numEpoch. often, other criteria are used to terminate the training: for example loss value, training accuracy, validation accuracy or combination. possible implementations:...

rzorzorzo

## Description currently, during training, especially with small batch size (1), prediction of a batch may be equal to the label. this results in zero loss which results in above...

rzorzorzo