Web Scraping with Google Apps Script

When you do web scraping with Google Apps Script (GAS), you may get the warning "To run this project, you need to grant access to your Google account. Please try again and grant permission." Here's how to deal with it.

1. Write the code for web scraping

Create a file to write the script in: "+" → "Script"

There are various ways to write scraping code; this time we'll use the following to fetch www.google.com.

function myFunction() {
  response = UrlFetchApp.fetch("https://www.google.com/");
  Logger.log(response);
}

2. Approve access permissions at run time

When you save the code, a dialog appears saying "Authorization required. This project requires access to your data." Click the "Review permissions" link.

A Google sign-in window opens, so click the account you want to use.

This app isn't verified by Google. The app is requesting access to private information in your Google account. Don't use this app until it has been verified by the developer and Google.

This window appears and is startling, but here you click "Advanced," then click "Go to ○○ project (unsafe)" that appears.

A window appears saying "is requesting access to your Google Account." To do web scraping you need to permit external access, so click the "Allow" button.

Running the script again confirmed that the target site's data could be retrieved.