doesn't work on window 10 21H2 version

[yfcck]

Caused by: java.lang.Exception: Exception while creating StreamOperatorStateContext. at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:255) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:268) at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:109) at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:711) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:687) at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.flink.util.FlinkException: Could not restore keyed state backend for CoBroadcastWithKeyedOperator_ac9a901f2ba35c2ad13a5f3044240476_(2/12) from any of the 1 provided restore options. at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160) at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:346) at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:164) ... 11 more Caused by: java.io.IOException: Could not load the native RocksDB library at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:882) at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:402) at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:93) at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:329) at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) ... 13 more Caused by: java.lang.UnsatisfiedLinkError: D:\checkpoint\rocksdb-lib-317ecc348d5fbe7c64fa10384c7eb8fc\librocksdbjni-win64.dll: librocksdbjni-win64.dll: A dynamic link library (DLL) initialization routine failed

not work with vc 2015/vc 2017 of win10 H21H2，

but it's working on win10 H20H2

flink v1.14.4

[fredia]

hi @wnnlyf, this looks a bit similar to https://github.com/facebook/rocksdb/issues/2531, could you please use a small Java program to try and load it to see if you can get more detailed error messages:

public class LoadTest {
  public static void main(final String args[]) {
    System.loadLibrary("librocksdbjni-win64");
  }
}

Then to compile and run it:

javac LoadTest.java
java -classpath . -Djava.library.path=C:\dir-containing-rocksdb-dll LoadTest

And one more question, the file librocksdbjni-win64.dll is often included within the Jar file(for me, it's located in frocksdbjni-6.20.3-ververica-1.0.jar), but your librocksdbjni-win64.dll is located in D:\checkpoint\rocksdb-lib-317ecc348d5fbe7c64fa10384c7eb8fc?

Could you please try loading xxxx.jar/librocksdbjni-win64.dll and D:\checkpoint\rocksdb-lib-317ecc348d5fbe7c64fa10384c7eb8fc\librocksdbjni-win64.dll: librocksdbjni-win64.dll separately? then see what happens.

[Myasuka]

@wnnlyf Flink does not support running on Windows with all features. And it would consume the maintainers of flink's forked RocksDB too much energy to support all platforms.

[yfcck]

frocksdbjni

hi @wnnlyf, this looks a bit similar to facebook/rocksdb#2531, could you please use a small Java program to try and load it to see if you can get more detailed error messages:

public class LoadTest {
  public static void main(final String args[]) {
    System.loadLibrary("librocksdbjni-win64");
  }
}

Then to compile and run it:

javac LoadTest.java
java -classpath . -Djava.library.path=C:\dir-containing-rocksdb-dll LoadTest

And one more question, the file librocksdbjni-win64.dll is often included within the Jar file(for me, it's located in frocksdbjni-6.20.3-ververica-1.0.jar), but your librocksdbjni-win64.dll is located in D:\checkpoint\rocksdb-lib-317ecc348d5fbe7c64fa10384c7eb8fc?

Could you please try loading xxxx.jar/librocksdbjni-win64.dll and D:\checkpoint\rocksdb-lib-317ecc348d5fbe7c64fa10384c7eb8fc\librocksdbjni-win64.dll: librocksdbjni-win64.dll separately? then see what happens.

still error : A dynamic link library (DLL) initialization routine failed

[shaoleiL]

I also had this problem when upgrading flink version from 1.14.6 to 1.18.1. Error reported as follows: java.lang.Exception: Exception while creating StreamOperatorStateContext. at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:258) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:256) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:753) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:728) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:693) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) ~[flink-runtime-1.18.1.jar:1.18.1] at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:922) [flink-runtime-1.18.1.jar:1.18.1] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) [flink-runtime-1.18.1.jar:1.18.1] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) [flink-runtime-1.18.1.jar:1.18.1] at java.lang.Thread.run(Thread.java:855) [?:1.8.0_412] Caused by: org.apache.flink.util.FlinkException: Could not restore keyed state backend for StreamFilter_20ba6b65f97481d5570070de90e4e791_(1/8) from any of the 1 provided restore options. at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:355) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:166) ~[flink-streaming-java-1.18.1.jar:1.18.1] ... 11 more Caused by: java.io.IOException: Could not load the native RocksDB library at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:990) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:920) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:449) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:100) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:338) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:355) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:166) ~[flink-streaming-java-1.18.1.jar:1.18.1] ... 11 more Caused by: java.lang.UnsatisfiedLinkError: C:\Users\xxx\AppData\Local\Temp\minicluster_d5c88fcd979a2a4985549c0dabf4946d\tm_0\tmp\rocksdb-lib-b3ff2d160e75effb257e99e2fc828c95\librocksdbjni-win64.dll: Can't find dependent libraries at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[?:1.8.0_412] at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2042) ~[?:1.8.0_412] at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1925) ~[?:1.8.0_412] at java.lang.Runtime.load0(Runtime.java:794) ~[?:1.8.0_412] at java.lang.System.load(System.java:1128) ~[?:1.8.0_412] at org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:102) ~[frocksdbjni-6.20.3-ververica-2.0.jar:?] at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:82) ~[frocksdbjni-6.20.3-ververica-2.0.jar:?] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:964) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.ensureRocksDBIsLoaded(EmbeddedRocksDBStateBackend.java:920) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:449) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend.createKeyedStateBackend(EmbeddedRocksDBStateBackend.java:100) ~[flink-statebackend-rocksdb-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:338) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:355) ~[flink-streaming-java-1.18.1.jar:1.18.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:166) ~[flink-streaming-java-1.18.1.jar:1.18.1] ... 11 more The development environment is windows 11 version 23H2. But in flink1.14.6 there is no problem.I don't know how to solve this problem. Please help me

[tuongthoai]

I encountered this problems on my window

my flink version is 1.18.1 and i have the same problems with @shaoleiL

i digged into the rabbit hole and found out that the code which copy the dll from the jar file to temp file was not working (copy 0 byte). As a result, the RocksDB cannot initialize properly.

I had tried to test to load a single file many people had guided but nothing work.

I hope anyone can fix this problem. Thank you.

[shaoleiL]

I encountered this problems on my window

my flink version is 1.18.1 and i have the same problems with @shaoleiL

i digged into the rabbit hole and found out that the code which copy the dll from the jar file to temp file was not working (copy 0 byte). As a result, the RocksDB cannot initialize properly.

I had tried to test to load a single file many people had guided but nothing work.

I hope anyone can fix this problem. Thank you.

Hi，@tuongthoai I solved this problem by adding the following dependencies to pom.xml.

<dependency>
    <groupId>org.rocksdb</groupId>
    <artifactId>rocksdbjni</artifactId>
    <version>6.29.5</version>
    <classifier>win64</classifier>
</dependency>

The Flink version should be compatible with the version of RocksDB, mainly the librocksdbjni-win64.dll file. However, this issue does not affect linux environments.I hope it helps.

[tuongthoai]

Hi thank you for your help. I just figure out the solution and tend to reply my question and help you to solve it too.

I simply use the latest version of rocksdb straight from facebook and it run without any error.

Thanks a lot @shaoleiL .