pyspark-ai

pyspark-ai copied to clipboard

Dataframe from webpage doesn't load entire table

1

When trying to load a dataframe from a webpage, e.g.: ```python df = spark_ai.create_df('https://en.wikipedia.org/wiki/List_of_countries_by_vehicles_per_capita') ``` It often doesn't load the entire table into the dataframe (the table on the webpage...

emlyn

ruff is faster.

MrPowers

Convert prompt.py into multiple files as per the target API

6

Currently, all the prompts are in one file. To improve the readability, we can have a new module "prompt" and split the file `prompt.py` into files such as `transform.py`, `plot.py`,...

gengliangwang

Add support for Llama2, Palm, Cohere, Anthropic, Replicate, Azure Models - using litellm

8

This PR adds support for 50+ models using liteLLM : https://github.com/BerriAI/litellm/ `ChatLiteLLM()` is integrated into langchain and allows you to call all models using the ChatOpenAI I/O interface https://python.langchain.com/docs/integrations/chat/litellm Here's...

ishaan-jaff

Support robust code generation with Code Llama

1

Tried with Code Llama and it fails with import code blocks. 

gengliangwang

Add an example notebook with Code Llama. For instance, we can try Code Llama on the following functions: - df.ai.plot() - df.ai.verify() - Create python UDF via `@spark_ai.udf` annotation

gengliangwang

Could not parse LLM output when using GPT3.5-turbo

1

When using GPT3.5-turbo with pyspark-ai, I'm getting the error below: NFO: Creating temp view for the transform: df.createOrReplaceTempView("spark_ai_temp_view_875e9a") > Entering new AgentExecutor chain... Traceback (most recent call last): File "/home/hadoop/p.py",...

s4saurabh

Hi, I'm having an issue using Vertex AI as LLM. This is the log: ``` import plotly.express as px df = spark.sql("SELECT Nationality, count(*) as cnt FROM football_stats GROUP BY...

chuyasturiano

Support PySpark Code generation for `transform` and improve usability.

1

This patch adds a new feature to the project that allows generating PySpark code instead of SQL code for a given prompt. This is valuable since it's closer to the...

grundprinzip

是否可以支持中文提交？

smileyboy2019