mapdb icon indicating copy to clipboard operation
mapdb copied to clipboard

expireStoreSize() is not working with fileDB

Open vipinmpd08 opened this issue 5 years ago • 3 comments

Hi, thanks for such a beautiful lightweight API. Just got one question related to fileDB expiry/overflow. My intention is to evict old items if the diskMap.db size becomes more than 500 MB in disk. But this doesnt seems to be working and I see the diskMap.db size continuously growing. Any suggestions ?

    val fileDB = DBMaker.fileDB("diskMap.db")
        .fileMmapEnable()
        .make()

    val diskMap = fileDB
        .hashMap("diskMap", Serializer.STRING, Serializer.BYTE_ARRAY)
        .expireStoreSize(500 * 1024 * 1024)
        .createOrOpen()

vipinmpd08 avatar Jun 17 '19 11:06 vipinmpd08

hello,do you meet this problem java.lang.NoSuchMethodError: org.mapdb.elsa.ElsaSerializerPojo.(I[Ljava/lang/Object;Ljava/util/Map;Ljava/util/Map;Ljava/util/Map;Lorg/mapdb/elsa/ElsaClassCallback;Lorg/mapdb/elsa/ElsaClassInfoResolver;)V

sunyaf avatar Jun 21 '19 03:06 sunyaf

I am facing the same problem, afaik expireStoreSize and expireMaxSize are not working for me for disk and memory db. I dont know what i am doing wrong. Here is the code which should terminate, but it is running until heap space is full...

        DB dbDisk = DBMaker
                .fileDB("mapdbfile")
                .fileMmapEnableIfSupported()
                .make();

        DB dbMemory = DBMaker
                .memoryDB()
                .make();


        HTreeMap onDisk = dbDisk
                .hashMap("onDisk")
                .expireStoreSize(50 * 1024 * 1024) // 100 MB
                .expireMaxSize(100)
                .expireExecutor(Executors.newScheduledThreadPool(2))
                .createOrOpen();

        HTreeMap inMemory = dbMemory
                .hashMap("inMemory")
                .expireStoreSize(10 * 1024 * 1024)
                .expireMaxSize(10)
                .expireOverflow(onDisk)
                .expireExecutor(Executors.newScheduledThreadPool(2))
                .create();


        inMemory.put("test", 2);

        System.out.println("" + inMemory.get("test"));

        int i = 0;
        while (onDisk.get("test") == null) {
            i++;
            System.out.println("not found... " + i + " " + inMemory.getSize() + " " + onDisk.size());
            byte[] bytes = new byte[1 * 1024 * 1024];
            new Random().nextBytes(bytes);
            inMemory.put(i, bytes);
            Thread.sleep(10000);
            System.out.println(" " + inMemory.getExpireMaxSize());
        }

        inMemory.close();
        onDisk.clear();
        onDisk.close();

pY4x3g avatar Nov 26 '19 15:11 pY4x3g

I found the solution, I thought that the eviction is always triggered after a put operation, but you have to activate the eviction process by using expireAfterCreate() without long which will activate the eviction after each put operation. Using it in addition with expireExecutor the eviction will be done by separate processes. I would like to trigger the eviction by myself but unfortunately the expireEvict() method is not working for me, but for the beginning the expireAfterCreate is sufficient for me. Would be great if this would be in the documentation since I had to dig through the code to understand the eviction process.

pY4x3g avatar Feb 24 '20 10:02 pY4x3g