What are the objectives of this library?

[k00ni]

Your question

What are the objectives of this library?

Its stated in the README.md: Because TransformersPHP is designed to be functionally equivalent to the Python library, it's super easy to learn from existing Python or Javascript code. Does that mean that you aim for a 99% coverage of the HuggingFace Python library? Or in other words: are you re-implementing the whole HuggingFace Python library in PHP?

Context (optional)

I recently started using HuggingFace and was surprised, that the PHP-support is so bad. Luckily, I found your library (saw your comment in HuggingFace forum).

I am wondering why there isn't a basic Python wrapper in PHP at least? My local tests show that its quiet possible to generate Python code on the fly for basic function calls at least. Of course, custom Python code would be hard to realize, but for common functionality its quiet doable.

The following script requires a working PHP 8 and Python 3 environment (with datasets, evaluate, torch, transformers[sentencepiece] installed via pip). Its very basic, but can read a PDF file using a third party lib and uses some Python-code to generate a summary of the PDF content.

use Smalot\PdfParser\Parser;

require __DIR__.'/vendor/autoload.php';

// get all text from a given PDF file
$parser = new Parser();
$document = $parser->parseFile('test.pdf');
$text = $document->getText();

// represents Python code (each entry is a line)
$pythonCode = [
    'from transformers import pipeline',
    'summarizer = pipeline("summarization", "sshleifer/distilbart-cnn-12-6")',
    'result = summarizer(',
    '"""',
    $text,
    '"""',
    ')',
    'print(result)',
];

$pythonFile = __DIR__.'/generated_file.py';
if (file_exists($pythonFile)) {
    unlink($pythonFile);
}

// write custom Python code to file for later execution
$pythonCode = implode(PHP_EOL, $pythonCode);
file_put_contents($pythonFile, $pythonCode);

// execute the Python file and save its output into $output
ob_start();
$output = shell_exec('python3 generated_file.py &2> /dev/null');
ob_end_clean();

var_dump($output);
// outputs [{'summary_text': '...

// get result as array
$resultAsArray = json_decode($output, true);

// process the result ...

Reference (optional)

No response