Hi, this is Theo from Apify, and today I'm
going to show you how to scrape Twitter without the Twitter API. Twitter updated
its free API policy and limited access to creation of up to 1 500 tweets per month:
That's approximately 50 tweets per day. That could be problematic for Twitter's most
active developers. With access to the paid free tier amounting to $100 per month, many developers
are looking for alternative solutions and more affordable solutions to the Twitter API. Apify’s
Twitter Scraper is one of those solutions. So in this short video tutorial, we're going to show you
how to use this tool. Now let's begin by creating an empty task because that way, we can save this
configuration and reuse it later. One option you have is to scrape by search term. You can add as
many as you like by clicking the plus add button, but we'll stick with the default input. You can
choose to filter tweets by content, you can limit the number of profiles, and set the maximum number
of tweets per search query. The default setting is 100, we'll just change it to 50. You have “add
user information” here as a default setting, the user information would be things like
account name, Twitter handle, followers, and so on. But bear in mind that does mean the
run would take longer than if it is disabled. You can scrape by Twitter profile or Twitter
URL, and that's your second option. You can again add as many as you like. So we can, for
example, add the Twitter handle for Elon Musk. You can also scrape replies in addition to tweets
(not just the tweets themselves). Tweets only, tweets, and replies to other users. You can also
scrape by Twitter URL, so in this case, we're going to put the URL for the ESPN NBA Twitter
page. You can also scrape specific dates. So, for example, we're going to scrape tweets from
the 1st of January till the 1st of February. In “Proxy configuration”, you can choose
your own proxies, but we recommend using automatic proxy for the sake of performance.
And you can extend the scraper functionality. So here we're going to extend the output function
to include the search term and search URL. So now that we're happy with our
settings, let's just click save and start. So the actor is running, you will start to see
results coming in while it's running, but it's not finished until the status changes to “Succeeded”.
So, the run has succeeded as you can see, we got 50 results in one minute. So now you can preview
or download your data by going to the “Storage” tab, and you can see it in any of the formats
provided. So let's take a look at it in JSON, and there we have it. That's it! Now you can
use the dates you've collected for spreadsheets or integrate it with other tools. If you're
interested in extracting even more data from the web, you can go to Apify Store for over a thousand
ready-made tools for web scraping and automation, including many tools for extracting specific
types of Twitter data such as hashtags, comments, and so on. Also, if you want to be part of a
like-minded community of web scraping enthusiasts, don't forget to join our Discord: link in
the description. if you enjoyed this video, don't forget to like and subscribe,
and click that bell, and check out our channel for other videos. That's all for
now, so long, and thanks for all the likes.
