data
data copied to clipboard
mev-fyi
The open-source repository mev.fyi aggregates research on Maximal Extractable Value (MEV). Explore curated academic papers, community contributions, and educational content on MEV and related topics.
### TODO - Update the [src/populate_csv_files/get_article_content/crawl_non_medium_websites.py](https://github.com/mev-fyi/src/populate_csv_files/get_article_content/crawl_non_medium_websites.py) to crawl all posts (URLs) from all websites in data.mev.fyi available at [`data/links/websites.csv`](https://github.com/mev-fyi/data/blob/main/data/links/websites.csv). Visualize websites at data.mev.fyi on _Websites_ tab. - Input: [website URLs](https://github.com/mev-fyi/data/blob/main/data/links/websites.csv)....
The Youtube data ingestion + indexing is big. It could be worthwhile to spin-off as a micro-service / package to make it more maintainable, have people fork it to build...
There are a lot of `awesome` github repos we can index/scrape and add to the database. Surely we can also automatically generate an "awesome of awesome" repos for MEV, DeFi,...
### TODO - Create a script in https://github.com/mev-fyi/data/blob/main/src/populate_csv_files/get_article_content/get_websites_from_articles.py where we extract the unique authors' blog link from all the articles from https://github.com/mev-fyi/data/blob/main/data/links/articles_updated.csv (`article` header). - Create a second script to...
Right now we manually add articles from websites. We can add a functionality to crawl all referenced websites and get the latest articles without manually adding them. Conversely, we can...
The current articles processing has copy pasted methods while we can simplify down with config json and adapting to a few edge cases. Most of the time it is mostly...
We can implement https://github.com/context-labs/autodoc to automatically generate documentations from github code repositories. That would be one step closer to make it easier to onboard developers to any protocol
Right now at every manual fetching of articles via `get_articles_content`, we overwrite the previous content with the latest fetched one, which propagates past edits. We would need to add safeguards...
Currently the Google Sheet of data.mev.fyi showing all data aggregates (articles, research papers, and so forth) isn't necessarily the nicest from a user experience perspective namely there is no user...