Auto-complete suggestions from Google and Bing are great to grasp popular or trending queries, helping you target additional keywords in the content, etc.
Yet wouldn't it be nice to get in them in bulk?
StreamSuggest retrieves these suggestions at scale, organises them neatly, visualises them in a tree or tabular form and exports them to CSV… For free!
I had much fun building it, so I’m really stoked to share it with the world today!
The tool was inspired by a discussion I had with Greg Bernhardt, Leszek Siemiński and Erlé Alberton earlier this year:
(follow these folks if you’re not already!)
I realised how flexible yet overlooked the Google Suggestions API was. Unlike other Google APIs, it’s super easy to use, there are no keys, no Oauth authentication... As Greg said: “It almost feels like a side project Google forgot about but keeps the lights on!”
So I rolled my sleeves up and gave these APIs the Streamlit treatment! 🤘
Below’s a quick tour of what the app does and how to use it.
Step 1: Setting things up!
First, type a keyword (e.g. 'SEO'):
Second, select your Search Engine:
Third, select the crawl depth:
Last, press 'Fetch Suggestions' to send your request to the API:
You'll be notified as soon as your results are retrieved:
Please be patient. There’s a sleep timer of ~1 second hard-coded into the app, as you would get blocked by Google otherwise!
Now here comes the fun part, checking the results! 🙌
Step 2: Check results in Tree View
The Tree view is fully dynamic, so you can click on each node to nest/expand each leaf
You can also save your preferred view as jpeg by right-clicking on the chart
Step 3: Check results in tabular form
The tabular view gathers all the scraped results (3 levels deep) and classifies them in 8 columns:
Root keyword
Level 01
Level 02
Level 03
Level 03 rank
Full string (All 3 levels)
Search engine (Bing or Google)
Date & time the data was crawled
You can also download a CSVs - just click the link!
Limitations & Support
StreamSuggest is still in Beta and can crawl up to 3 levels deep (2 levels in Tree view), which is about ~450 keywords in one go… Not a bad start! And I’m planning to add more levels soon! 🙌
For the curious, here’s the stack I used:
Language: Python (heck, what did you expect here?! :))
Web framework: Streamlit
Tree view: ECharts.js via Streamlit Components
Deployment: Heroku
Kudos to Fanilo for porting Echarts.js to Streamlit, I couldn't have created StreamSuggest without it!
Drop me a line if questions, bugs or suggestions!
Lastly, this app is free and should remain that way. Buy me a coffee if it’s useful to you! 🙏
Happy scraping!