Problems Trying to use with Laravel on AWS

[coogle]

System Info

Ubuntu 22.02 on AWS running on a m5.4xlarge instance. The code is running in the context of a Laravel application (specifically a mixin for Illuminate\Support\Str), being tested via artisan tinker

PHP Version

8.2

Environment/Platform

• [X] Command-line application
• [ ] Web application
• [ ] Serverless
• [ ] Other (please specify)

Description

I am trying to do some simple usage with Laravel on AWS via artisan tinker and I am running into a problem where the first execution works, but then the second execution hangs. Here is the simple code I'm executing (technically I'm executing it via a mixin to Str and running it in artisan tinker like so:

$ ./artisan tinker

> \Illuminate\Support\Str::namedEntities('What kind of campgrounds are near New York City?');
= [
    [
      "entity_group" => "LOC",
      "score" => 0.99945451815923,
      "word" => "New York City",
      "start" => null,
      "end" => null,
    ],
  ]
>

where namedEntities() just executes the following:

class PredictNamedEntities
{
    public function namedEntities(): callable
    {
        return function (
            string $input,
            AggregationStrategy $aggregationStrategy = AggregationStrategy::MAX,
            string $model = 'Xenova/bert-base-NER',
        ): array {

            $ner = pipeline(
                Task::Ner,
                $model,
            );

            return array_values(
                $ner(
                    $input,
                    aggregationStrategy: $aggregationStrategy
                )
            );
        };
    }
}

I put print statements between the pipeline and the $ner command to see where it was freezing up and it's happening when I call $ner(). Like I said this is only on the 2nd execution -- the first execution works perfectly as expected.

I have tried multiple different types of instances (I was actually trying to benchmark what kind of instance sizes I might need to use this library in a production setting) and it's the same behavior across them all (including a m5.4xlarge with 16 CPUs and 64G of RAM available). All of them freeze up like this.

I also execute this exact same type of command from artisan tinker very often on my Macbook M1 and it works fine, so this is something specifically happening at least in AWS or perhaps Intel architecture vs. Apple Silicon.

This sort of feels like an FFI issue on the surface, especially since it works perfectly (and fast) the very first call but then the second call stalls out... perhaps you have a different view?

Reproduction

1. Create an instance in AWS
2. Attempt to run a named entity recognition via artisan tinker
3. Get a resullt back
4. Replay the exact same line of code a 2nd time
5. Hangs