Workflow icon indicating copy to clipboard operation
Workflow copied to clipboard

How to configure persistent storage?

Open singpolyma opened this issue 9 years ago • 3 comments
trafficstars

If I want to store state in, say, Redis instead of in files, can that be done?

singpolyma avatar Mar 13 '16 18:03 singpolyma

Unfortunately no. For performance reasons the write/read is to the filesystem, since workflow performs appends to the log file.

But the files are located in the .tcachedata folder. that folder can be uploaded/downloaded from to any other storage before/after the execution.

agocorona avatar Mar 13 '16 21:03 agocorona

What if I change to do the writes only manually so I doesn't need to be appending constantly?

singpolyma avatar Mar 13 '16 21:03 singpolyma

It is hard coded to write in the file system anyway. It would be far more easy to change it to write anywhere, but the unloading of the log from memory and appending to the log would be lost.

All the code is in Stat.hs

Just removing the instance IResource Stat part

and adding:

instance Indexable Stat where
   key s@Stat{wfName=name}=  statPrefix ++ name
   key (Running _)= keyRunning

instance Serializable Stat where
  serialize= pack . show
  deserialize= read . unpack

Then define Persist, which contains the three routines for writing /reading/deleting bytestrings in the storage. filePersist is the Persist by default. It can be the example for creating any other persistence.

Once defined, setDefaultPersist set this new persistence and from this moment on all Serializable instances are written/read this way.

agocorona avatar Mar 13 '16 22:03 agocorona