Sam Stoelinga

OffsiteMiddleware not working

2

I saw the request is replaced with dont_filter=True, if I remove that the spider will just stop when it gets to the same url. I need to use the offsite...

The captcha part wasn't working and was showing broken gif images on linux. This patch shows the captcha code in a small window by using the PIL library. The code...

Missing log4j dependency for SURF with scala/sbt

5

I'm trying to extract Surf features without success. It seems it's missing a log4j dependency. The code that I'm using: ``` import de.lmu.ifi.dbs.jfeaturelib.features.SURF import ij.process.ColorProcessor; import java.io.File; import java.io.IOException; import...

Support Weaviate Auto Schema

2

Currently the class has to be precreated in Weaviate for the Spark Connector to work. However, it might be possible to rely on auto schema creation instead, but this would...

pyspark with scala 2.13 jar doesn't work

1

This error will be thrown if you use the 2.13 fat jar with pyspark: ``` E py4j.protocol.Py4JJavaError: An error occurred while calling o50.save. E : java.lang.NoSuchMethodError: 'scala.collection.convert.AsScalaExtensions$ListHasAsScala scala.jdk.CollectionConverters$.ListHasAsScala(java.util.List)' E at...

Use memory safe functions

1

@sam-h-bean reported a potential memory leak and this is a blind shot in potentially fixing this. It uses memory safe functions to get the data and makes a copy of...

Support writing a dataframe that only contains subset of Weaviate class

1

Assume you have Weaviate class: ``` Article (title: String, content: String) ``` However your Spark dataframe also has an `author` column then I should still be able to write a...

https://spark-packages.org/

Ability to issue or configure an API key inside KubeAI config. Users should be able to secure kubeai by requiring an OpenAI API key. This should be compatible with using...