[JS] RAG reranking throws TypeError following sample code from docs

[morganric]

Describe the bug A clear and concise description of what the bug is. include full error messages.

I am trying to implement RAG reranking within my firebase functions application using Genkit, following this article: https://firebase.google.com/docs/genkit/rag

I consistently hit this error:

Reranking failed with correct format: TypeError: Cannot read properties of undefined (reading 'map')

 at rerank (/Users/richardmorgan/Documents/dev/neuro/mvp/functions/node_modules/@genkit-ai/ai/lib/reranker.js:123:29)

To Reproduce Steps to reproduce the behavior. Code samples.

here is an example of my code:

try { console.log("Starting reranking process...");

    // Check if docs is valid
    if (!docs || !Array.isArray(docs) || docs.length === 0) {
      console.error("docs is invalid:", docs);
      throw new Error("Invalid docs array for reranking");
    }

    // Log the structure of the first document to debug
    console.log("First doc structure:", JSON.stringify(docs[0], null, 2));

    // Create a simpler query
    const queryText = aiPrompt.trim().substring(0, 500);
    // Limit length to be safe
    console.log("Using query text:", queryText);

    // Try reranking with direct document passing
    const rerankedDocs = await ai.rerank({
      reranker: semanticRanker512,
      query: queryText, // Try with plain text instead of Document object
      documents: docs, // Pass the retrieved docs directly
    });

    console.log("Reranking successful!");
    console.log("Number of reranked docs:", rerankedDocs.length);

    // Check the structure of the reranked docs
    if (rerankedDocs.length > 0) {
      console.log("First reranked doc structure:",
        JSON.stringify(rerankedDocs[0], null, 2));
    }

    // Include the reranked docs in the final response
    res.status(200).json({
      data: {
        success: true,
        message: "Mood assessment submitted successfully",
        timestamp: data.timestamp,
        aiRecommendations: aiResponse,
        aiDocs: docs,
        rerankedDocs: rerankedDocs,
      },
      status: "success",
    });
  } catch (error) {
    console.error("Detailed reranking error:", error);

    // Try another approach with explicit document conversion
    try {
      console.log("Attempting alternative reranking approach...");

      // Convert docs to a simpler format that the reranker might accept
      const simpleDocs = docs.map((doc) => {
        return {
          id: doc.metadata?.id ||
          `doc_${Math.random().toString(36).substring(7)}`,
          content: [{
            text: typeof doc.content === "string" ?
              doc.content :
              JSON.stringify(doc.content),
          }],
        };
      });

      console.log("Created simplified docs:", simpleDocs.length);

      // Try reranking with the simplified docs
      const altRerankedDocs = await ai.rerank({
        reranker: semanticRanker512,
        query: aiPrompt,
        documents: simpleDocs,
      });

      console.log("Alternative reranking successful!");

      // Include these alternative results
      res.status(200).json({
        data: {
          success: true,
          message: "Mood assessment submitted successfully",
          timestamp: data.timestamp,
          aiRecommendations: aiResponse,
          aiDocs: docs,
          rerankedDocs: altRerankedDocs,
        },
        status: "success",
      });
    } catch (secondError) {
      console.error("Both reranking approaches failed:", secondError);

      // Return response without the reranked docs
      res.status(200).json({
        data: {
          success: true,
          message: "Mood assessment submitted success (reranking failed)",
          timestamp: data.timestamp,
          aiRecommendations: aiResponse,
          aiDocs: docs,
        },
        status: "success",
      });
    }
  }

Expected behavior A clear and concise description of what you expected to happen.

I expect a response containing a ranked array of docs with scores.

Screenshots If applicable, add screenshots to help explain your problem.

Runtime (please complete the following information):

• OS: [e.g. Linux, MacOS]
• Version [e.g. 22]

MacOS 12.5

** Node version

• run node --version at paste here v22.2.0

Additional context Add any other context about the problem here.

[hkaras19]

@morganric, were you able to find a solution to this? Currently running into it as well.

Note: when following the docs directly, I also get this error:

Error: Unable to resolve the reranker
    at rerank (/layers/google.nodejs.yarn/yarn_modules/node_modules/@genkit-ai/ai/lib/reranker.js:120:11)
    at process.processTicksAndRejections (node:internal/…

const query = "example query..."

const rerankedDocuments = await ai.rerank({
      reranker: 'vertexai/semantic-ranker-512',
      query,
      documents,
});

[trolloks]

I am also running into the same issue as @hkaras19 . I've tried implementing it according to the Vertex Reranker Test App in the repo to no avail as well.

Init

const reranker = genkit({
	plugins: [
		vertexAI({
			location: LOCATION,
			projectId: PROJECT_ID,
			googleAuth: {
				scopes: ['https://www.googleapis.com/auth/cloud-platform'],
			},
		}),
                vertexAIRerankers({
			location: LOCATION,
			projectId: PROJECT_ID,
			rerankers: ['vertexai/semantic-ranker-512'],
		}), // reranking model
	],
});

Query

const documents = embeddings.map((e) => Document.fromText(e.text));
const rerankedDocuments = await reranker.rerank({
	reranker: 'vertexai/semantic-ranker-512',
	query: Document.fromText(input),
	documents,
});

[hectoritr]

Same issue here! unable to resolve reranker, following the reranker example from this repo with no luck! I've tried using the vertexAireranker plugin to get the reranker from there, and the error message changed to ""Cannot read properties of undefined (reading 'map')""

There is a PR on this repo that might lead to something! https://github.com/firebase/genkit/pull/1259/files

@trolloks please comment if you fix it

[hectoritr]

@trolloks I confirm that using it like in that PR works.

Two days trying to understand what was happening!

[trolloks]

@hectoritr Thanks for the update!

[morganric]

@hkaras19 no sorry, looks like @trolloks and @hectoritr have made progress..

[muddylemon]

Has anyone found a workaround for this? I've tried to match the PR referenced above, but adding rerankOptions to the vertexAI plugin raises a typescript error that rerankOptions does not exist in type 'PluginOptions'

export const ai = genkit({
  plugins: [
    googleAI({ apiKey: GEMINI_API_KEY }),
    vertexAI({ projectId: PROJECT_ID, location: 'global' }),
    vertexAIRerankers({
      location: 'global',
      rerankers: ['vertexai/semantic-ranker-512@latest'],
      projectId: PROJECT_ID,
    }),
  ],
});

  let rerankedResponse;
  try {
    rerankedResponse = await ai.rerank({
      reranker: 'vertexai/semantic-ranker-512@latest',
      query: Document.fromText(query),
      documents: retrievedDocs.map((doc) => Document.fromText(doc.content.map((c) => c.text).join('\n'))),
    });
  } catch (error) {
    console.error('Error during reranking:', error);
    throw new Error('Reranking failed');
  }

Consistently errors with:

Error during reranking: Error: Unable to resolve the reranker
    at rerank (/workspace/node_modules/@genkit-ai/ai/lib/reranker.js:125:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async reranker (/workspace/lib/core/reranker.js:70:28)
    at async /workspace/lib/rag-lookup.js:76:26
    at async /workspace/node_modules/@genkit-ai/core/lib/action.js:142:27
    at async /workspace/node_modules/@genkit-ai/core/lib/tracing/instrumentation.js:75:24
    at async runInNewSpan (/workspace/node_modules/@genkit-ai/core/lib/tracing/instrumentation.js:60:10)
    at async actionFn.run (/workspace/node_modules/@genkit-ai/core/lib/action.js:101:18)
    at async /workspace/node_modules/firebase-functions/lib/v2/providers/https.js:234:32
    at async /workspace/node_modules/firebase-functions/lib/common/providers/https.js:544:26

[diego-lipinski-de-castro]

Same problem using the local dev store and retriever

"@genkit-ai/dev-local-vectorstore": "^1.19.3"

const retriever = devLocalRetrieverRef("index");

const ai = genkit({
	plugins: [
		openAI(),
		devLocalVectorstore([
			{
				indexName:"index",
				embedder: openAI.embedder("text-embedding-3-small"),
			},
		]),
	],
});

const results = await ai.retrieve({
	retriever: retriever,
	query: query,
	options: {
		k: options.k,
	},
});

const rerankedResults = await ai.rerank({
	reranker: retriever,
	query: query,
	documents: results,
});

[morganric]

I've moved on from my initial error (I think I was incorrectly importing the reranker), however I now see:

Error message: Unable to resolve the reranker

Full error: Error: Unable to resolve the reranker at rerank (...functions/node_modules/@genkit-ai/ai/lib/reranker.js:120:11)

I am not sure I have enabled the service correctly in GCP portal