ERROR :- There can only be 256 non-closed Scanner instances. Create them once per thread!

[hariraogotit]

Hi Team,

I used the code in https://github.com/gliwka/hyperscan-java#use-of-the-patternfilter and I got - "There can only be 256 non-closed Scanner instances. Create them once per thread!"

I had multiple threads running the above code and I had one instance of PatternFilter per thread. Upon using filter.close(); fixed the issue. I am no longer getting the error.

Hope this is how I am expected to code/fix a multithreaded application?

[hariraogotit]

Hi Team,

I used the code in https://github.com/gliwka/hyperscan-java#use-of-the-patternfilter as follows

public class Task implements Runnable{
    @Override
    public void run() {
        System.out.println("Begin");
        List<Pattern> patterns = asList(
                Pattern.compile("The number is ([0-9]+)", Pattern.CASE_INSENSITIVE),
                Pattern.compile("The color is (blue|red|orange)")
                // and thousands more
        );

//not thread-safe, create per thread
        PatternFilter filter = null;
        try {
            filter = new PatternFilter(patterns);
        } catch (CompileErrorException compileErrorException) {
            compileErrorException.printStackTrace();
        }

//this list now only contains the probably matching patterns, in this case the first one
        List<Matcher> matchers = filter.filter("The number is 7 the NUMber is 27");
        filter.close();
//now we use the regular java regex api to check for matches - this is not hyperscan specific
        for(Matcher matcher : matchers) {
            while (matcher.find()) {
                // will print 7 and 27
                System.out.println(matcher.group(1));
            }
        }

        System.out.println("End");
    }

Then ran it like below

 for(int i=0; i<=8000 ; i++) {
               Runnable runnable = new Task();
               Thread thread = new Thread(runnable);
               thread.start();
           }

I get

Exception in thread "Thread-1535" java.lang.RuntimeException: There can only be 256 non-closed Scanner instances. Create them once per thread! at com.gliwka.hyperscan.wrapper.Scanner.(Scanner.java:40) at com.gliwka.hyperscan.util.PatternFilter.(PatternFilter.java:61)

Kindly help.

FYI :- If I do not close (filter.close(); ) in proper place I get - Scratch space has already been deallocated

[jasonparallel]

The counter for the number of scanners is not thread safe. My PR fixes this https://github.com/gliwka/hyperscan-java/pull/141

[xiaochy2]

Hey Team, I have an API service that creates PatternFilters and do regex job for each http requests. I met this error and want to ask the best the practice for fixing that. Should I close PatternFilters at the end of each requests? And what can I do if the server received more than 256 requests at the same time as there is a limit of # of scanners?

[gliwka]

This error has been fixed in the latest release.