Add a filter (or setting?) to allow alt text generation when the site has basic authentication

[JiveDig]

Is your enhancement related to a problem? Please describe.

Right now alt text generation fails when basic authentication is enabled because OpenAI (or whichever provider) can't access the image directly.

This code works specifically for my use-case, but I'm wondering if there is a way to more easily handle this while we're working on protected staging sites.

/**
 * Handle basic auth for Classifai alt text generation via OpenAI.
 *
 * @param array  $options The options.
 * @param string $url     The URL.
 * @param string $feature The feature.
 *
 * @return array
 */
add_filter( 'classifai_openai_api_request_post_options', function( array $options, string $url, string $feature ): array {
	// Bail if not our feature.
	if ( 'classifai_feature_descriptive_text_generator' !== $feature ) {
		return $options;
	}

	// Bail if no body.
	if ( ! isset( $options['body'] ) ) {
		return $options;
	}

	// Bail if no messages.
	$body = json_decode( $options['body'], true );
	if ( ! $body || empty( $body['messages'] ) ) {
		return $options;
	}

	// Set the username and password.
	$un = 'someun';
	$pw = 'somepw';

	// Encode the credentials.
	$creds = base64_encode( $un . ':' . $pw );

	// Loop through the messages.
	foreach ( $body['messages'] as &$message ) {
		// Bail if not an array.
		if ( ! is_array( $message['content'] ) ) {
			continue;
		}

		// Loop through the items.
		foreach ( $message['content'] as &$item ) {
			// Bail if not an image.
			if ( empty( $item['type'] ) || 'image_url' !== $item['type'] ) {
				continue;
			}

			// Fetch the image manually.
			$resp = wp_remote_get( $item['image_url']['url'], [
				'headers' => [ 'Authorization' => 'Basic ' . $creds ],
			] );

			// Bail if error.
			if ( is_wp_error( $resp ) ) {
				continue;
			}

			// Get the raw body.
			$raw  = wp_remote_retrieve_body( $resp );

			// Encode the body.
			$b64  = base64_encode( $raw );

			// Figure out the MIME type (fallback to jpeg).
			$info = @getimagesizefromstring( $raw );
			$mime = $info ? $info['mime'] : 'image/jpeg';

			// Replace with a Data URI.
			$item = [
				'type'      => 'image_url',
				'image_url' => [
					'url'    => "data:{$mime};base64,{$b64}",
					'detail' => $item['image_url']['detail'] ?? 'auto',
				],
			];
		}
	}

	// Encode the body.
	$options['body'] = wp_json_encode( $body );

	return $options;

}, 10, 3  );

Designs

No response

Describe alternatives you've considered

No response

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[dkotter]

The approach you're taking works fine though you could use the classifai_chatgpt_descriptive_text_request_body filter to be slightly more efficient (would allow you to remove a couple checks you have now). But otherwise, what you have is the right approach.

In regards to a filter or setting, I guess do you have thoughts on how best that should work? I see benefit in adding a filter around the image URL, allowing you to more easily change that to something else (in this case, Base64-encoded data).

But not sure on a specific setting or filter that helps bypass basic auth. I guess the idea would be a filter/setting where you can set your basic auth credentials and then we automatically download the image and Base64-encode it?

[JiveDig]

I guess the idea would be a filter/setting where you can set your basic auth credentials and then we automatically download the image and Base64-encode it?

Yes, this is what I was thinking. This way, no matter what provider/agent we're using, and whichever setting/feature would require it, it would just work with a single filter.

Something like this:

add_filter( 'classifai_basic_auth_credentials', function( $credentials ) {
	$credentials = [
		'username' => 'someun',
		'password' => 'somepw',
	];
	
	return $credentials;
} );

Or separate like this:

add_filter( 'classifai_basic_auth_username', function( $username ) {
	return 'someun';
} );

add_filter( 'classifai_basic_auth_password', function( $password ) {
	return 'somepw';
} );

The tool/plugin is super useful and would often be running on staging environments that are protected, so the less custom configuration we need in terms of getting around basic auth, the better. IMHO :)