Add example: improving embeddings sequences with lowpass filtering

[jagilley]

Hey team - here's an example notebook for using signal processing methods to improve the OpenAI embeddings' ability to encode long documents. Full paper can be found here: https://jagilley.github.io/fft-embed.html

Let me know if you have any suggested changes!

[ted-at-openai]

Thanks for the contribution. I tried running your submission and got an error on the 4th cell:

---------------------------------------------------------------------------
ParameterError                            Traceback (most recent call last)
[/var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_14653/3454959581.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_14653/3454959581.py) in 
     23 # get FFTs
     24 print('Applying FFTs...')
---> 25 all_embeddings_paras_fft = [get_fft(embedding) for embedding in tqdm(all_embeddings_paras)]
     26 
     27 if apply_lowpass:

[/var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_14653/3454959581.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_14653/3454959581.py) in (.0)
     23 # get FFTs
     24 print('Applying FFTs...')
---> 25 all_embeddings_paras_fft = [get_fft(embedding) for embedding in tqdm(all_embeddings_paras)]
     26 
     27 if apply_lowpass:

[/var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_14653/3454959581.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_14653/3454959581.py) in get_fft(embedding)
      4 # get FFTs
      5 def get_fft(embedding):
----> 6     return librosa.stft(embedding, n_fft=32, win_length=4)
      7 
      8 # lowpass filter

[~/.virtualenvs/openai/lib/python3.9/site-packages/librosa/core/spectrum.py](https://file+.vscode-resource.vscode-cdn.net/Users/ted/code/openai-cookbook/examples/~/.virtualenvs/openai/lib/python3.9/site-packages/librosa/core/spectrum.py) in stft(y, n_fft, hop_length, win_length, window, center, dtype, pad_mode)
    215 
    216     # Check audio is valid
--> 217     util.valid_audio(y)
    218 
    219     # Pad the time series so that frames are centered

[~/.virtualenvs/openai/lib/python3.9/site-packages/librosa/util/utils.py](https://file+.vscode-resource.vscode-cdn.net/Users/ted/code/openai-cookbook/examples/~/.virtualenvs/openai/lib/python3.9/site-packages/librosa/util/utils.py) in valid_audio(y, mono)
    291 
    292     if mono and y.ndim != 1:
--> 293         raise ParameterError(
    294             "Invalid shape for monophonic audio: "
    295             "ndim={:d}, shape={}".format(y.ndim, y.shape)

ParameterError: Invalid shape for monophonic audio: ndim=2, shape=(6, 1536)

[ted-at-openai]

I also see two of the packages aren't used - could clean them up from the imports:

• openai
• pickle

[ted-at-openai]

Also, is it common to refer to a sliding window over the document as the time domain? If it anything, it feels like a spatial dimension more than the time dimension. I worry folks reading the first paragraph might get confused on what this is about. Suggest adding a bit more to the intro that's more beginner-friendly (while still keeping the rest).

[ted-at-openai]

One more suggestion:

• add error bars (binomial standard error = sqrt(p*(1-p)/n)) so that people know whether the differences are statistically significant or not

[ted-at-openai]

Haven't heard back, so will close this for now.