Are you a social media marketer? Do you want to track your competitor’s Facebook page? Or, are you a social media researcher looking to extract publically available data on a Facebook page? This tutorial is for you!
This tutorial uses Facebook Graph API, a legal way of mining Facebook data, to extract info from public pages. Through the Graph API, you can download Facebook page posts and comments to Excel.
If you don’t have time to do this by hand and want the best tool for the job, you can sign up for a free trial, no credit card required, with Quintly, this cloud-based Facebook Scraper Tool.
It’s simple to use and has a lot of functionalities. You can also use it to scrape Instagram, YouTube, Google+, Twitter, LinkedIn, and Pinterest.
The tutorial will start with simple Facebook API calls using your browser, then will transition to automation using Python script. This might sound technical, but no worries- no programming knowledge is required! Later, I will introduce you a cloud-based Facebook scraper tool I use to save time.
Part 1: Scraping Facebook Page posts and Comments with URL queries
Step 1: Get a Facebook Graph API Access Token
To begin, you must get an access token from the Facebook Graph API explorer. Just log in to your regular Facebook account to get the access token which has a lifespan of two hours.
Step 2: Scrapping Facebook Page posts with URL queries
To scrape Facebook Page Posts do the following query with your access token obtained from step 1.
Let us use Barack Obama page as an example. Some Facebook pages don’t have a short clean URL like the one for Barack Obama page does. It might be something like “https://www.facebook.com/Funny-Dog-Videos-691496887527705/”, which uses “691496887527705” as a page name. For now, we’ll stick with the simpler URL of Obama’s page.
To extract posts from Barack Obama page since 01-January-2018 to 30-April-2018, we copy:
You can replace the “barackobama” page name and access token with yours.
Below is the result we get from above query:
You can copy and convert the returned JSON to CSV as below.
Also, you can click “before” and “after” cursors to move through a thread of results but as you can see the process is a bit tedious, This is where automation comes into place using Python.
Step 3: Scraping Facebook Page comments with URL queries
You can download comments for a specific post using below query:
You can get the post-ID from the Excel sheet downloaded in the previous step #2 and access token from step #1.
The query will look like this:
You can copy the above comments and use JSON to CSV converter to get them into CSV.
We can harvest data by directly pasting URLs into the browser, but it gets tedious if you want to get more data. Also, there are some limitations with this method.
In the next part, I will use Python to automate the process.
Part 2: Scraping Facebook Page Posts and Comments with Python
Don’t worry if you don’t know anything about programming; I have a working script which I’m going to use in the tutorial. You can use it, too.
Step 1: Install Python 3 and a Code Editor
Go to the Python website and install Python 3. Follow this guide on how to install Python correctly on Windows, Linux, and Mac. In the guide, Atom Code editor is recommended, but personally, I use Notepadd++ for Windows.
If you have followed the guide correctly below is what you should see in Windows:
Step 2: Edit and Save the Python code on your computer
If you have followed the previous steps, you should have three things: the access token, the Facebook page to scrape, and the preferred period.
If your access token has expired (they expire in two hours), just click the get token button again to get a new one.
From the below script, replace everything within the quotation marks with your own code: “page_id”, “since_date”,”until_dare”,and “access_token”.
Click to View Code below. You can copy the code and paste it on your editor and save it to a folder as a .py file which we will access later from the command line.
Step 3: Scraping Facebook Posts with Reactions data count
Here is an example of how I have saved the Python code on my computer. You can name the file anything you want, make sure the file has the .py extension.
Now open your command line program, PowerShell, on Windows. Navigate to the folder with your downloaded script, which I described in the previous steps.
Once you are there, type “python scriptname.py” to run the script and start scraping. For me, it is python scrape_posts.py
As you can see, it seems that Obama hasn’t posted much— only seven posts since the beginning of 2018 to end of April.
The script saves the output CSV file in the same folder as the Python code.
The output file content looks like the image below:
Step 4: Scraping Facebook Comments
Now that we have scraped the Facebook posts, it’s time to get the comments associated with the posts. Make sure the output file from the previous step is still intact in the same folder.
Below is a new code for you to use when scraping. You use this code the same way as the previous one. If you don’t want to get comments on some posts, you can delete the posts in the Excel sheet obtained from the last step. This new script reads that Excel file.
The procedure to run the code is same as in the previous step, input the same Facebook page name on the “file_id” field and your “access _token” used in the last step.
Click to View Code below. You can copy the code and paste it on your editor and save it to a folder as a .py file which we will access later from the command line. I save my code as “scrape_comments.py”
On the command line navigate to the folder with the script and posts CSV extracted from the previous step and run the script.
Those seven posts we extracted earlier have 3465 comments!
The output file content looks like below.
Part 3: Scraping Facebook Page posts and Comments with Quintly a Cloud-based Facebook Data Scraper
As I mentioned earlier, the above procedures are not for everyone. They are very raw, time-consuming, and involve some manual work.
For example, if you want to be updated on the performance of the posts or download new comments on a daily basis, then you would have to redo the work every single day.
One of the tools I use and recommend for scraping Facebook posts, comments and other social media platforms is Quintly. It’s a cloud-based Scraper that works 24/7 for you, calling the APIs and aggregating the data in one interface.
They have a 14-day free trial.
Let’s continue with our previous example, this time using Quintly.
Step 1: Search for a Profile, Select and add it
To search for Facebook, Instagram, RSS Feed or Pinterest profiles, insert the entire URL link into the search box.
Barack Obama profile added on my dashboard.
I’m on a Medium package, and I can add up to 15 profiles of either Facebook, Twitter, Instagram, Google+, Youtube, LinkedIn, and Pinterest.
Step 2: Scrape Facebook Posts by adding “Own Posts Table” metric
In our case, we want posts reactions. So I have added “Own Posts Table” metric and set the date range from January 1, 2018, to April 30, 2018.
Then you can export Facebook posts to Excel
The output file content looks like below.
Step 2: Scrape Facebook Posts Comments by adding “Comments Table” metric
Since we want posts comments, I have added the “Comments Table” metric and set the date range from January 1, 2018 to April 30, 2018.
Then you can export Facebook comments into Excel
The output file content looks like below.
As you can see, this method is very effective! If you want to save time and improve your business or research, I recommend you sign up for Quintly. You can scrape Facebook automatically, as well as other social media pages on Twitter, LinkedIn, Youtube, Google+, Pinterest, and Instagram.
Hope you have enjoyed the tutorial! If you have any questions, don’t hesitate to drop a comment.
Frequently Asked Questions
Can you scrape Facebook Group Emails?
Scraping Facebook emails or emails left in Facebook groups is not supported by the Facebook API.
Can you scrape Facebook Profiles?
Scraping Facebook profiles is not supported by the Facebook API.
Can you scrape Facebook Groups?
Yes, it’s possible to scrape Open Public Groups with the API but not Private Groups even if you are a member or an admin