langflow icon indicating copy to clipboard operation
langflow copied to clipboard

Unable to upload file to folder

Open jordanrfrazier opened this issue 1 year ago • 6 comments

Describe the bug Unable to upload file to folder due to key error when selecting folder_name from the `flow. I see that a recent commit has made it such that each flow will have a default folder name, if none exists, but this not backwards-compatible with existing flows made in <=v1.0.0a38. Also - I would have expected the payload to send my folder that I want the flow created in, not for it to create a folder in my store.

(EDIT: I see that that recent change was just for making sure the flow was in a folder, but does not export that folder name in the json, so that's an invalid concern. Seems we just may need to pass the selected folder to the /folders/upload endpoint)

https://github.com/langflow-ai/langflow/blob/543e8d52afbb8e64ae22255909f4453484b2bb07/src/backend/base/langflow/api/v1/folders.py#L194

Browser and Version

  • Browser [e.g. chrome, safari] firefox
  • Version [e.g. 22] v1.0.0a38

To Reproduce Steps to reproduce the behavior:

  1. pip install
  2. run
  3. create folder
  4. upload a flow
  5. see logs

logs

                             │ langflow/.venv/lib/python3.11/site-packages/langflow/api/v1/folders.py:209 in upload_file                                  │                 
                             │                                                                                                                                                                                      │                 
                             │   206 │                                                                                                                                                                              │                 
                             │   207 │   folder_results = session.exec(                                                                                                                                             │                 
                             │   208 │   │   select(Folder).where(                                                                                                                                                  │                 
                             │ ❱ 209 │   │   │   Folder.name == data["folder_name"],                                                                                                                                │                 
                             │   210 │   │   │   Folder.user_id == current_user.id,                                                                                                                                 │                 
                             │   211 │   │   )                                                                                                                                                                      │                 
                             │   212 │   )                                                                                                                                                                          │                 
                             ╰──────────────────────────────────────────────────────────    
                             KeyError: 'folder_name'  

Fix(?)

  • Pass currently selected folder to payload when uploading
  • Use that as the folder name in /folders/upload
  • Add test for /folders/upload

jordanrfrazier avatar May 28 '24 18:05 jordanrfrazier

To fix the KeyError when uploading a file to a folder due to the recent commit that made each flow have a default folder name, you need to ensure that the folder_name and folder_description keys are present in the uploaded data. Additionally, you can use the utility function create_default_folder_if_it_doesnt_exist to handle the creation of a default folder if it doesn't exist.

Here is the relevant part of the code that handles the upload:

@router.post("/upload/", response_model=List[FlowRead], status_code=201)
async def upload_file(
    *,
    session: Session = Depends(get_session),
    file: UploadFile = File(...),
    current_user: User = Depends(get_current_active_user),
):
    """Upload flows from a file."""
    contents = await file.read()
    data = orjson.loads(contents)

    if not data:
        raise HTTPException(status_code=400, detail="No flows found in the file")

    if "folder_name" not in data or "folder_description" not in data:
        raise HTTPException(status_code=400, detail="Missing folder_name or folder_description in the data")

    folder_results = session.exec(
        select(Folder).where(
            Folder.name == data["folder_name"],
            Folder.user_id == current_user.id,
        )
    )
    existing_folder_names = [folder.name for folder in folder_results]

    if existing_folder_names:
        data["folder_name"] = f"{data['folder_name']} ({len(existing_folder_names) + 1})"

    folder = FolderCreate(name=data["folder_name"], description=data["folder_description"])

    new_folder = Folder.model_validate(folder, from_attributes=True)
    new_folder.id = None
    new_folder.user_id = current_user.id
    session.add(new_folder)
    session.commit()
    session.refresh(new_folder)

    del data["folder_name"]
    del data["folder_description"]

    if "flows" in data:
        flow_list = FlowListCreate(flows=[FlowCreate(**flow) for flow in data["flows"]])
    else:
        raise HTTPException(status_code=400, detail="No flows found in the data")
    # Now we set the user_id for all flows
    for flow in flow_list.flows:
        flow.user_id = current_user.id
        flow.folder_id = new_folder.id

    return create_flows(session=session, flow_list=flow_list, current_user=current_user)

Additionally, you can use the create_default_folder_if_it_doesnt_exist function to ensure a default folder is created if it doesn't exist:

def create_default_folder_if_it_doesnt_exist(session: Session, user_id: UUID):
    folder = session.exec(select(Folder).where(Folder.user_id == user_id)).first()
    if not folder:
        folder = Folder(name=DEFAULT_FOLDER_NAME, user_id=user_id, description=DEFAULT_FOLDER_DESCRIPTION)
        session.add(folder)
        session.commit()
        session.refresh(folder)
        session.exec(
            update(Flow)  # type: ignore
            .where(
                and_(
                    Flow.folder_id == None,  # type: ignore # noqa
                    Flow.user_id == user_id,
                )
            )
            .values(folder_id=folder.id)
        )
        session.commit()
    return folder

This ensures that the necessary keys are present and prevents the KeyError, while also handling the creation of a default folder if needed [1][2].

Additionally, you can refer to the uploadFlowsFromFolders function in the frontend service to ensure the correct API endpoint and data format are used when uploading flows:

export async function uploadFlowsFromFolders(
  flows: FormData
): Promise<FlowType[]> {
  try {
    const response = await api.post(`${BASE_URL_API}folders/upload/`, flows);

    if (response?.status !== 201) {
      throw new Error(`HTTP error! status: ${response?.status}`);
    }
    return response.data;
  } catch (error) {
    console.error(error);
    throw error;
  }
}

This function handles the API call to upload flows to the server, ensuring that the data is correctly formatted and the appropriate endpoint is used [3].

Furthermore, the useFolderStore in the frontend ensures that folders are managed correctly, including handling the default folder and starter projects:

import { create } from "zustand";
import { DEFAULT_FOLDER, STARTER_FOLDER_NAME } from "../constants/constants";
import {
  getFolderById,
  getFolders,
  uploadFlowsFromFolders,
} from "../pages/MainPage/services";
import { FoldersStoreType } from "../types/zustand/folders";
import useFlowsManagerStore from "./flowsManagerStore";

export const useFolderStore = create<FoldersStoreType>((set, get) => ({
  folders: [],
  getFoldersApi: (refetch = false) => {
    return new Promise<void>((resolve, reject) => {
      if (get()?.folders.length === 0 || refetch === true) {
        get().setLoading(true);
        getFolders().then(
          (res) => {
            const foldersWithoutStarterProjects = res.filter(
              (folder) => folder.name !== STARTER_FOLDER_NAME,
            );

            const starterProjects = res.find(
              (folder) => folder.name === STARTER_FOLDER_NAME,
            );

            set({ starterProjectId: starterProjects!.id ?? "" });
            set({ folders: foldersWithoutStarterProjects });

            const myCollectionId = res?.find(
              (f) => f.name === DEFAULT_FOLDER,
            )?.id;

            set({ myCollectionId });

            if (refetch === true) {
              useFlowsManagerStore.getState().refreshFlows();
              useFlowsManagerStore.getState().setAllFlows;
            }

            get().setLoading(false);
            resolve();
          },
          () => {
            set({ folders: [] });
            get().setLoading(false);
            reject();
          },
        );
      }
    });
  },
  setFolders: (folders) => set(() => ({ folders: folders })),
  loading: false,
  setLoading: (loading) => set(() => ({ loading: loading })),
  getFolderById: (id) => {
    get().setLoadingById(true);
    if (id) {
      getFolderById(id).then(
        (res) => {
          const setAllFlows = useFlowsManagerStore.getState().setAllFlows;
          setAllFlows(res.flows);
          set({ selectedFolder: res });
          get().setLoadingById(false);
        },
        () => {
          get().setLoadingById(false);
        },
      );
    }
  },
  selectedFolder: null,
  loadingById: false,
  setLoadingById: (loading) => set(() => ({ loadingById: loading })),
  getMyCollectionFolder: () => {
    const folders = get().folders;
    const myCollectionId = folders?.find((f) => f.name === DEFAULT_FOLDER)?.id;
    if (myCollectionId) {
      getFolderById(myCollectionId).then((res) => {
        set({ myCollectionFlows: res });
      });
    }
  },
  setMyCollectionFlow: (folder) => set(() => ({ myCollectionFlows: folder })),
  myCollectionFlows: null,
  setMyCollectionId: () => {
    const folders = get().folders;
    const myCollectionId = folders?.find((f) => f.name === DEFAULT_FOLDER)?.id;
    if (myCollectionId) {
      set({ myCollectionId });
    }
  },
  myCollectionId: "",
  folderToEdit: null,
  setFolderToEdit: (folder) => set(() => ({ folderToEdit: folder })),
  folderUrl: "",
  setFolderUrl: (url) => set(() => ({ folderUrl: url })),
  folderDragging: false,
  setFolderDragging: (folder) => set(() => ({ folderDragging: folder })),
  folderIdDragging: "",
  setFolderIdDragging: (id) => set(() => ({ folderIdDragging: id })),
  uploadFolder: () => {
    return new Promise<void>(() => {
      const input = document.createElement("input");
      input.type = "file";
      input.onchange = (event: Event) => {
        if (
          (event.target as HTMLInputElement).files![0].type ===
          "application/json"
        ) {
          const file = (event.target as HTMLInputElement).files![0];
          const formData = new FormData();
          formData.append("file", file);
          uploadFlowsFromFolders(formData).then(() => {
            get().getFoldersApi(true);
          });
        }
      };
      input.click();
    });
  },
  starterProjectId: "",
  setStarterProjectId: (id) => set(() => ({ starterProjectId: id })),
}));

This ensures that the frontend correctly handles folder management, including the creation and uploading of flows to the appropriate folders [4].

Details

About Dosu This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

dosubot[bot] avatar May 28 '24 19:05 dosubot[bot]

I see. The upload button on the main page is specifically for uploading folders, not flows.

Perhaps just a quick json validation check is enough to avoid confusion here

jordanrfrazier avatar May 29 '24 06:05 jordanrfrazier

Is there a way to upload a Flow directly? I've been creating a blank project, then importing, but that doesn't update the Name or Description of the Flow. Can we have a direct "upload flow" option?

jordanrfrazier avatar Jun 04 '24 15:06 jordanrfrazier

To address your concerns and provide a solution for directly uploading a Flow, you can implement a direct "upload flow" option that allows for the Name and Description of the Flow to be updated accordingly.

Backend: Add an Endpoint for Uploading a Single Flow

You can add a new endpoint to handle the upload of a single flow. This endpoint will ensure that the Name and Description are included in the uploaded data:

@router.post("/upload_flow/", response_model=FlowRead, status_code=201)
async def upload_single_flow(
    *,
    session: Session = Depends(get_session),
    file: UploadFile = File(...),
    current_user: User = Depends(get_current_active_user),
):
    """Upload a single flow from a file."""
    contents = await file.read()
    data = orjson.loads(contents)

    if not data:
        raise HTTPException(status_code=400, detail="No flow found in the file")

    if "name" not in data or "description" not in data:
        raise HTTPException(status_code=400, detail="Missing name or description in the data")

    flow = FlowCreate(name=data["name"], description=data["description"], user_id=current_user.id)
    new_flow = Flow.model_validate(flow, from_attributes=True)
    new_flow.id = None
    session.add(new_flow)
    session.commit()
    session.refresh(new_flow)

    return new_flow

Frontend: Add a Function to Handle the Upload of a Single Flow

Next, you can add a function in the frontend to handle the API call for uploading a single flow:

export async function uploadSingleFlow(flow: FormData): Promise<FlowType> {
  try {
    const response = await api.post(`${BASE_URL_API}upload_flow/`, flow);

    if (response?.status !== 201) {
      throw new Error(`HTTP error! status: ${response?.status}`);
    }
    return response.data;
  } catch (error) {
    console.error(error);
    throw error;
  }
}

Frontend: Add a Button and Handler for Uploading a Single Flow

Finally, you can add a button and handler in your frontend to allow users to upload a single flow directly:

import React from 'react';
import { uploadSingleFlow } from '../services';

const UploadFlowButton = () => {
  const handleUpload = async (event: React.ChangeEvent<HTMLInputElement>) => {
    const file = event.target.files?.[0];
    if (file && file.type === "application/json") {
      const formData = new FormData();
      formData.append("file", file);
      try {
        const uploadedFlow = await uploadSingleFlow(formData);
        console.log("Uploaded Flow:", uploadedFlow);
      } catch (error) {
        console.error("Error uploading flow:", error);
      }
    }
  };

  return (
    <input type="file" accept="application/json" onChange={handleUpload} />
  );
};

export default UploadFlowButton;

This setup allows you to upload a single flow directly, ensuring that the Name and Description are updated as specified in the uploaded JSON file. This should streamline the process and avoid the need to create a blank project first.

To continue talking to Dosu, mention @dosu.

dosubot[bot] avatar Jun 04 '24 16:06 dosubot[bot]

@jordanrfrazier to import a specific flow you need to create a new blank one, from the dropdown click "Import" and pass it the JSON file.

The "upload" button in the main page is for folders (I've sent this improvement https://github.com/langflow-ai/langflow/pull/2125)

nicoloboschi avatar Jun 10 '24 16:06 nicoloboschi

Hi @jordanrfrazier

Do you need any assistance with this case? If not, please let us know if this issue can be closed.

carlosrcoelho avatar Jul 17 '24 20:07 carlosrcoelho

Thank you for your contribution! This issue will be closed. If you have any questions or encounter another problem, please open a new issue and we will be ready to assist you.

carlosrcoelho avatar Jul 22 '24 01:07 carlosrcoelho

@jordanrfrazier to import a specific flow you need to create a new blank one, from the dropdown click "Import" and pass it the JSON file.

The "upload" button in the main page is for folders (I've sent this improvement #2125)

tried this, nothing imported.....

it seems , the only way is export the folder, and import the folder, this works

maxpaynebupt avatar Aug 09 '24 04:08 maxpaynebupt