Volumes not save flows

[Wilton]

Processors are not saved after to and restart the containers, whenever I see the volume database_repository it is modified. I need to create the flows and they stay on the volume. And I have a question regarding clusters volumes, how should I implement them? Should I make a volume for each instance or can I just create a shared volume for all instances?

[xemuliam]

Docker containers should be stateless. Thus all container-related parameters (configuration, processors etc.) should be managed on docker image build stage. But, certainly, you can mount your custom processor as volume

Flow management is a pain-point for NiFi. Here you have 2 options:

• mark flow.xml.gz as volume
• use NiFi-Registry subproject

Easiest way to work with flow which is stored outside of container:

1. Create folder with NiFi configuration (use your own local path)

docker run --name nifi -d xemuliam/nifi && \
docker cp nifi:/opt/nifi/conf /Users/xemuliam/Projects/Docker/nifi_test && \
docker rm -f nifi && \
ls -la /Users/viacheslav.kalashnikov/Projects/Docker/nifi_test/conf

2. Run NiFi with folder from previous step mounted as volume docker run --name nifi -d -p 8080 -v /Users/xemuliam/Projects/Docker/nifi_test/conf:/opt/nifi/conf xemuliam/nifi
3. Make some changes in flow
4. Drop container docker rm -f nifi
5. Run fresh NiFi container with mounted configuration folder docker run --name nifi -d -p 8080 -v /Users/xemuliam/Projects/Docker/nifi_test/conf:/opt/nifi/conf xemuliam/nifi
6. Voila! Your changes are there!

[xemuliam]

Regarding cluster

the best way is to store all node-related information (flow, settings etc.) on separate places. this approach will let you avoid any issues and contention for file-based resources (e.g. flow)

[sylnsr]

@xemuliam I am following your approach. I started with copying only flow.xml.gz and nifi.properties, then starting up a new container mapping just these 2 files. The container will not start however. It has these messages in the log:

sed: cannot rename /opt/nifi/nifi-current/conf/sedTNxUtT: Device or resource busy
replacing target file  /opt/nifi/nifi-current/conf/nifi.properties

and

Caused by: java.nio.file.FileSystemException: ./conf/flow.xml.gz: Device or resource busy

[sylnsr]

Also .. if the flow data is in flow.xml.gz then whats the point of the flowfile_repository folder?