olmocr icon indicating copy to clipboard operation
olmocr copied to clipboard

I am receiving WARNING - Attempt *: All connection attempts failed

Open likith1908 opened this issue 9 months ago • 17 comments

(olmocr) jupyter@v:~/Test/olmocr$  python -m olmocr.pipeline ./localworkspace --pdfs /home/jupyter/Test/olmocr/tests/mypdfs/test_page_08_1224001.pdf
INFO:olmocr.check:pdftoppm is installed and working.
2025-03-04 05:36:48,006 - __main__ - INFO - Got --pdfs argument, going to add to the work queue
2025-03-04 05:36:48,007 - __main__ - INFO - Loading file at /home/jupyter/Test/olmocr/tests/mypdfs/test_page_08_1224001.pdf as PDF document
2025-03-04 05:36:48,008 - __main__ - INFO - Found 1 total pdf paths to add
Sampling PDFs to calculate optimal length: 100%|██████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 149.50it/s]
2025-03-04 05:36:48,016 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00
INFO:olmocr.work_queue:Found 1 total paths
INFO:olmocr.work_queue:0 new paths to add to the workspace
2025-03-04 05:36:48,280 - __main__ - INFO - Starting pipeline with PID 536
2025-03-04 05:36:48,280 - __main__ - INFO - Downloading model 'allenai/olmOCR-7B-0225-preview'
Fetching 15 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 20295.02it/s]
2025-03-04 05:36:48,543 - __main__ - INFO - Model download complete 'allenai/olmOCR-7B-0225-preview'
INFO:olmocr.work_queue:Initialized local queue with 1 work items
2025-03-04 05:36:48,596 - __main__ - WARNING - Attempt 1: All connection attempts failed
2025-03-04 05:36:49,609 - __main__ - WARNING - Attempt 2: All connection attempts failed
2025-03-04 05:36:50,621 - __main__ - WARNING - Attempt 3: All connection attempts failed
2025-03-04 05:36:51,633 - __main__ - WARNING - Attempt 4: All connection attempts failed
2025-03-04 05:36:52,647 - __main__ - WARNING - Attempt 5: All connection attempts failed
2025-03-04 05:36:53,659 - __main__ - WARNING - Attempt 6: All connection attempts failed
2025-03-04 05:36:54,695 - __main__ - WARNING - Attempt 7: All connection attempts failed

Why is this happening?

likith1908 avatar Mar 04 '25 05:03 likith1908

I meet the same problem!

Image

chaoStart avatar Mar 04 '25 07:03 chaoStart

(olmocr) jupyter@v:~/Test/olmocr$  python -m olmocr.pipeline ./localworkspace --pdfs /home/jupyter/Test/olmocr/tests/mypdfs/test_page_08_1224001.pdf
INFO:olmocr.check:pdftoppm is installed and working.
2025-03-04 05:36:48,006 - __main__ - INFO - Got --pdfs argument, going to add to the work queue
2025-03-04 05:36:48,007 - __main__ - INFO - Loading file at /home/jupyter/Test/olmocr/tests/mypdfs/test_page_08_1224001.pdf as PDF document
2025-03-04 05:36:48,008 - __main__ - INFO - Found 1 total pdf paths to add
Sampling PDFs to calculate optimal length: 100%|██████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 149.50it/s]
2025-03-04 05:36:48,016 - __main__ - INFO - Calculated items_per_group: 500 based on average pages per PDF: 1.00
INFO:olmocr.work_queue:Found 1 total paths
INFO:olmocr.work_queue:0 new paths to add to the workspace
2025-03-04 05:36:48,280 - __main__ - INFO - Starting pipeline with PID 536
2025-03-04 05:36:48,280 - __main__ - INFO - Downloading model 'allenai/olmOCR-7B-0225-preview'
Fetching 15 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 20295.02it/s]
2025-03-04 05:36:48,543 - __main__ - INFO - Model download complete 'allenai/olmOCR-7B-0225-preview'
INFO:olmocr.work_queue:Initialized local queue with 1 work items
2025-03-04 05:36:48,596 - __main__ - WARNING - Attempt 1: All connection attempts failed
2025-03-04 05:36:49,609 - __main__ - WARNING - Attempt 2: All connection attempts failed
2025-03-04 05:36:50,621 - __main__ - WARNING - Attempt 3: All connection attempts failed
2025-03-04 05:36:51,633 - __main__ - WARNING - Attempt 4: All connection attempts failed
2025-03-04 05:36:52,647 - __main__ - WARNING - Attempt 5: All connection attempts failed
2025-03-04 05:36:53,659 - __main__ - WARNING - Attempt 6: All connection attempts failed
2025-03-04 05:36:54,695 - __main__ - WARNING - Attempt 7: All connection attempts failed

Why is this happening?

Do you solve it?

chaoStart avatar Mar 04 '25 07:03 chaoStart

i have same problem,on ubuntu 24.05

Image

EndOfTime avatar Mar 04 '25 07:03 EndOfTime

I got an error after waiting for a few minutes: 2025-03-05 11:00:34,738 - main - WARNING - Attempt 299: All connection attempts failed 2025-03-05 11:00:35,765 - main - WARNING - Attempt 300: All connection attempts failed 2025-03-05 11:00:36,765 - main - INFO - Got cancellation request for SGLang server Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/opt/aiProject/projects/olmocr/olmocr/pipeline.py", line 1064, in asyncio.run(main()) File "/opt/software/miniconda3/envs/olmocr/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/opt/software/miniconda3/envs/olmocr/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/software/miniconda3/envs/olmocr/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/opt/aiProject/projects/olmocr/olmocr/pipeline.py", line 1042, in main await sglang_server_ready() File "/opt/aiProject/projects/olmocr/olmocr/pipeline.py", line 649, in sglang_server_ready raise Exception("sglang server did not become ready after waiting.") Exception: sglang server did not become ready after waiting.

Did not receive any update from the team! @jakep-allenai - could you please look into this issue?

likith1908 avatar Mar 04 '25 09:03 likith1908

You just need to wait a bit longer for sglang to initialize, sometimes it takes 3-4 minutes to startup, depending on your setup.

jakep-allenai avatar Mar 04 '25 16:03 jakep-allenai

You just need to wait a bit longer for sglang to initialize, sometimes it takes 3-4 minutes to startup, depending on your setup.

Okay, will let you if it works!! Thanks

likith1908 avatar Mar 04 '25 16:03 likith1908

You just need to wait a bit longer for sglang to initialize, sometimes it takes 3-4 minutes to startup, depending on your setup.

ProcessLookupError: ERROR:asyncio:Task exception was never retrieved future: <Task finished name='Task-2' coro=<sglang_server_host() done, defined at /app/olmocr/olmocr/pipeline.py:614> exception=SystemExit(1)> Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/app/olmocr/olmocr/pipeline.py", line 1081, in asyncio.run(main()) File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/base_events.py", line 673, in run_until_complete self.run_forever() File "/usr/lib/python3.12/asyncio/base_events.py", line 640, in run_forever self._run_once() File "/usr/lib/python3.12/asyncio/base_events.py", line 1992, in _run_once handle._run() File "/usr/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/app/olmocr/olmocr/pipeline.py", line 627, in sglang_server_host sys.exit(1) SystemExit: 1

Hello, I have a similar issue, but the error message is not exactly the same. Could you please help me with this? Thank you!

Devcode518 avatar Mar 05 '25 07:03 Devcode518

You need to wait as long as it took for you to download the 7B model from HuggingFace. Even if the program exits early due to excessive waiting time, at this point, you should observe your OS's download speed to ensure that the model has finished downloading (because huggingface maintains high speed when downloading models, at least that's what I experience.). Run the command line again; good luck!

Image|500x300

Of course, you can also download the model from HuggingFace in advance and specify the path using command-line arguments. (I haven't tried it)

--model MODEL         List of paths where you can find the model to convert this pdf. You can specify several different paths here, and the script will try to use the one which is fastest to access

bwnjnOEI avatar Mar 05 '25 12:03 bwnjnOEI

I get "sglang server did not become ready after waiting" error after 300 attemts. I run it again after some time, still doesn't work. I tried downloading the model and use --model tag, it doesn't work.

GasperNLP avatar Mar 07 '25 18:03 GasperNLP

I get "sglang server did not become ready after waiting" error after 300 attemts. I run it again after some time, still doesn't work. I tried downloading the model and use --model tag, it doesn't work.

I found the problem. Somehow I used the wrong version of python when executing. python3 ... instead of just python. And I had olmocr installed in both, but not the sglang.

GasperNLP avatar Mar 08 '25 12:03 GasperNLP

@GasperNLP could you specify which Python version are you using ?

alexcumplido avatar Mar 18 '25 13:03 alexcumplido

I think it should be 3.11. But i used pip to install olmocr and sglang, not pip3 and therefore it was installed in python not python3.

GasperNLP avatar Mar 18 '25 14:03 GasperNLP

Interesting, definitely recommend using a conda environment: https://www.anaconda.com/docs/getting-started/miniconda/main

jakep-allenai avatar Mar 18 '25 15:03 jakep-allenai

What would change if we use or not use conda environment?

likith1908 avatar Mar 18 '25 15:03 likith1908

Pip's dependency solver is not awesome, your system python might have had some other packages installed and that's getting a different final set of packages than what you would get if you installed clean. With a conda environment you would have a clean environment just how we test locally.

jakep-allenai avatar Mar 18 '25 15:03 jakep-allenai