Some trained Flux models have state_dict keys starting with `model.diffusion_model.` causing error in loading

[hablaba]

I noticed this when trying to load a non base flux dev model from CivitAI to train a Lora on top of. This basic script seemed to help convert it:

from safetensors.torch import load_file, save_file

dict = load_file("<model_path>", device="cuda")
keys = dict.keys()
fixed= {k.replace("model.diffusion_model.", ""): dict[k] for k in keys}

print(fixed.keys())
save_file(fixed, "<output_model_path")

Perhaps you have a better way of fixing this but figured I'd at least provide the script for posterity :)

[dosubot[bot]]

Hello @thinley4! I'm here to help you with any bugs, questions, or contributions you have. Let's work together to resolve the issue you're facing.

To resolve the TypeError indicating that prompt.invoke is not a function, ensure that you are correctly importing and using the ChatPromptTemplate class. The invoke method is defined within the ChatPromptTemplate class, so the error might be due to an incorrect import or usage.

Here is an example of how to correctly import and use the ChatPromptTemplate class:

import { ChatOpenAI } from "@langchain/openai";
import { createOpenAIFunctionsAgent, AgentExecutor } from "langchain/agents";
import { pull } from "langchain/hub";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const model = new ChatOpenAI({
  openAIApiKey: "sk-XXXX",
});

async function main() {
  const prompt = await pull<ChatPromptTemplate>(
    "hwchase17/openai-functions-agent"
  );

  const agent = await createOpenAIFunctionsAgent({
    llm: model,
    prompt,
    tools: []
  });

  const agentExecutor = new AgentExecutor({
    agent,
    tools: [],
  });

  console.log(agentExecutor);
}

main();

Ensure that the ChatPromptTemplate class is correctly imported from the @langchain/core/prompts module and that you are calling the invoke method on an instance of ChatPromptTemplate. Additionally, make sure that the pull function is correctly used to fetch the prompt template. Note that the import paths have been updated to reflect the new module structure as per the latest changes in the langchain library ^[1][2][3].

To continue talking to Dosu, mention @dosu.