Afnan retweetledi

Here is an open-source tool to generate a complete dataset.
1. Describe the data you want
2. An orchestrator agent searches the web
3. Sub-agents run in parallel to fetch the data
4. You get a structured dataset you can download
For example, you can run Bigset with the query "all leica lenses being sold on amazon", or "leica stores in kyoto with their opening hours and ratings".
Bigset uses @Tiny_Fish's free Search and Fetch APIs in the background. You can configure it to refresh the data on a schedule.
You can self-host it with your own keys.
Here is the GitHub repository: github.com/tinyfish-io/bi….
You can get free TinyFish API keys here: bit.ly/4u4JvE1
Thanks to the TinyFish team for partnering with me on this post.
TinyFish@Tiny_Fish
What if you and your agent had all the data that always stays fresh? Structured, on demand, never stale. Introducing BigSet. Describe the data you need in plain English → get a structured dataset built from the live web, that refreshes regularly. It's live and open-source.
English










