Cómo extraer reseñas de Google Maps

Andrei Ogiolan on Apr 21 2023

Introducción

Google Maps es uno de los servicios de mapas y navegación más utilizados del mundo, que proporciona a los usuarios una forma sencilla de encontrar y explorar lugares, empresas y puntos de interés. Una de las principales características de Google Maps es la posibilidad de buscar lugares y ver información detallada sobre ellos, como su ubicación, reseñas, fotos y mucho más.

Scraping this data from Google Maps can be useful for businesses to track and analyze the performance of their locations, for researchers to study patterns in consumer behavior and for individuals to find and explore new places.

The purpose of this article is to provide a step-by-step guide on how to scrape Google Maps Reviews with our API using Node.js. We will cover everything from setting up the development environment to extracting relevant data and discussing potential issues. By the end of this article, you will have the knowledge and tools you need to scrape Google Maps place results on your own.

¿Por qué deberías utilizar un rascador profesional en lugar de construir el tuyo?

Utilizar un scraper profesional puede ser una mejor opción que crear uno propio por varias razones. En primer lugar, los scraper profesionales están diseñados para manejar una amplia variedad de tareas de scraping y están optimizados para el rendimiento, la fiabilidad y la escalabilidad. Están diseñados para manejar grandes cantidades de datos y pueden manejar varios tipos de sitios web y tecnologías web. Esto significa que los raspadores profesionales a menudo pueden extraer datos con mayor rapidez y precisión que un raspador personalizado.

Además, los raspadores profesionales suelen venir con funciones integradas como la resolución de CAPTCHA, la rotación de IP y la gestión de errores, que pueden hacer que el proceso de raspado sea más eficiente y menos propenso a errores. También ofrecen soporte y documentación que puede ser útil cuando se enfrenta a cualquier problema.

Otro aspecto importante es que los proveedores de scrapers profesionales cumplen las políticas de scraping de los sitios web que scrapean y pueden proporcionar un uso legal de los datos, lo que es importante tener en cuenta a la hora de scrapear datos.

Finally, in our particular case, in order to scrape Google Maps Reviews, for best results, you need to pass a data_id parameter to your Google URL. This parameter usually looks something like this : 0x87c0ef253b04093f:0xafdfd6dc1d3a2b4e. I know this may sound intimidating at first as you may have no idea how to get the data_id property for a specific place and you are right, because Google hides this information and it is not visible on the page when you are searching for a place in Google Maps. But, fortunately, using a professional scraper like ours takes care of that by finding this data for you. We will talk in the later sections about how to get the data_id and how to scrape Google Maps reviews using our API.

Definir nuestro objetivo

What are Google Maps reviews?

Google Maps reviews are the ratings and comments left by users on Google Maps about a specific place. These reviews include information such as the user's name, the date the review was left, the rating given, and the review text.

Scraping Google Maps reviews can be useful for businesses who want to track and analyze the performance of their locations, researchers who want to study patterns in consumer behavior, and individuals who want to find and explore new places. By extracting the reviews data, businesses can identify the strengths and weaknesses of their locations, and make improvements accordingly. Researchers can study the sentiment of the reviews and find patterns in consumer behavior. Individuals can also use this information to make decisions about where to go and what to do.

¿Cómo es nuestro objetivo?

Puesta en marcha

Before beginning to scrape Google Maps reviews, it's important to have the necessary tools in place. The primary requirement is Node.js, a JavaScript runtime that enables the execution of JavaScript on the server-side, which can be downloaded from their official website. Additionally, an API KEY is required, which can be obtained by creating an account here and activating the SERP service.

Después de configurar Node.js y obtener una CLAVE de API, el siguiente paso es crear un archivo de script Node.js. Esto se puede hacer ejecutando el siguiente comando:

$ touch scraper.js 

Y ahora pegue la siguiente línea en su archivo:

console.log("¡Hola Mundo!")

Y ejecute el siguiente comando:

$ node scraper.js

If you see the message "Hello World!" displayed on the terminal, it indicates that Node.js has been successfully installed and you are ready to proceed to the final step. This final step is to obtain the Place ID of the place you are interested in scraping reviews. This is where our API comes in handy, it's easy to use and doesn't require any additional libraries to be installed.

En primer lugar, en un archivo js necesitas importar el módulo incorporado `https` de Node.js para poder enviar peticiones a nuestra API. Esto se puede hacer de la siguiente manera:

const https = require("https");

En segundo lugar, debe especificar su clave API, un término de búsqueda y las coordenadas del lugar que le interesa:

const API_KEY = "<YOUR-API-KEY-HERE>" // You can get by creating an account - https://app.webscrapingapi.com/register

const query = "Waldo%20Pizza"

const coords = "@38.99313451901278,-94.59368586441806"

Tip: This is how you get the coordinates for a place on Google Maps:

The next step is to include the obtained Place ID in an options object, to let our API know which location's reviews you want to scrape:

const options = {

  "method": "GET",

  "hostname": "serpapi.webscrapingapi.com",

  "port": null,

  "path": `/v1?engine=google_maps&api_key=${API_KEY}&type=search&q=${query}&ll=${coords}`,

  "headers": {}

};

A continuación, debe configurar una llamada a nuestra API con toda esta información:

const req = https.request(options, function (res) {

 const chunks = [];

 res.on("data", function (chunk) {

   chunks.push(chunk);

 });

 res.on("end", function () {

   const body = Buffer.concat(chunks);

   const response = JSON.parse(body.toString());

   const data_id = response.place_results.data_id;

   if (data_id) {

       console.log(data_id);

   }

   else {

       console.log('We could not find a data_id property for your query. Please try using another query')

   }

 });

});

req.end();

Lastly, you can execute the script you have just created and wait for the results to be returned:

$ node scraper.js

Y deberías obtener de vuelta la propiedad data_id impresa en la pantalla:

$ ​​0x87c0ef253b04093f:0xafdfd6dc1d3a2b4es

That concludes the setup process, with the data_id property, now you have all the necessary information to create a scraper for Google Maps reviews using our API using Node.js.

Let’s start scraping Google Reviews

With the environment set up, you are ready to begin scraping Google Maps Reviews with our API. To proceed, you need to set up the data parameter as previously mentioned. With all the necessary information available, you can set up the data_id parameter as follows:

const data_id = "0x87c0ef253b04093f:0xafdfd6dc1d3a2b4e" // the data_id we retrieved earlier

Now, the only thing left to do is to modify the options object, thus telling our API that you would like to scrape reviews from Google Maps:

const options = {

  "method": "GET",

  "hostname": "serpapi.webscrapingapi.com",

  "port": null,

  "path": `/v1?engine=google_maps_reviews&api_key=${API_KEY}&data_id=${data_id}`, // there is no need in having a query anymore, data_id is enough to identify a place

  "headers": {}

};

And this is everything you need to do. Your script should now look like this:

const http = require("https");

const API_KEY = "<YOUR-API-KEY-HERE>"

const data_id = "0x87c0ef253b04093f:0xafdfd6dc1d3a2b4e" // the data_id we retrieved earlier

const options = {

  "method": "GET",

  "hostname": "serpapi.webscrapingapi.com",

  "port": null,

  "path": `/v1?engine=google_maps_reviews&api_key=${API_KEY}&data_id=${data_id}`, // there is no need in having a query anymore, data_id is enough to identify a place

  "headers": {}

};

const req = http.request(options, function (res) {

  const chunks = [];

  res.on("data", function (chunk) {

    chunks.push(chunk);

  });

  res.on("end", function () {

    const body = Buffer.concat(chunks);

    const response = JSON.parse(body.toString())

    console.log(response);

  });

});

req.end();

After executing this script, you should receive a response that appears similar to this:

reviews: [

    {

      link: 'https://www.google.com/maps/reviews/data=!4m8!14m7!1m6!2m5!1sChZDSUhNMG9nS0VJQ0FnSUMyem9pOEdBEAE!2m1!1s0x0:0xafdfd6dc1d3a2b4e!3m1!1s2@1:CIHM0ogKEICAgIC2zoi8GA%7CCgwI1vuBkwYQiKeWyQE%7C?hl=en-US',

      date: '8 months ago',

      rating: 5,

      snippet: 'Wow, if you have dietary restrictions this is absolutely the place to go! Both for the variety of restrictions they cater to as well as the taste of the dishes.The good: great tasting food. Very conscious of dietary restrictions which include multiple types of vegan cheeses as well as gluten free. Decent drink selection.The meh: service is nice but a touch slow. Maybe understaffed? Prices are average for pizzas.The bad: noneFeatures: Did not see any masks on anyone inside. Unsure of cleaning practices so I cannot speak to that.Dine in: Yes\n' +

        'Takeout: Yes\n' +

        'Curbside pickup: YesWow, if you have dietary restrictions this is absolutely the place to go! Both for the variety of restrictions they cater to as well as the taste of the dishes. ...More',

      likes: 3,

      user: [Object],

      images: [Array]

    },

    {

      link: 'https://www.google.com/maps/reviews/data=!4m8!14m7!1m6!2m5!1sChZDSUhNMG9nS0VJQ0FnSURXOUxHSUl3EAE!2m1!1s0x0:0xafdfd6dc1d3a2b4e!3m1!1s2@1:CIHM0ogKEICAgIDW9LGIIw%7CCgwI3OnIkQYQwLGL1gM%7C?hl=en-US',

      date: '9 months ago',

      rating: 5,

      snippet: "We love Waldo Pizza! We have dairy allergies and Waldo offers a wide range of vegan cheeses as well as a ton of different toppings. The vegan dessert here is always excellent as well, super rich in flavor. Of course the traditional pizza, pasta and dessert are also amazing! It's great to have both options under one roof!Dine in: Yes\n" +

        'Outdoor seating: No ...More',

      likes: 1,

      user: [Object],

      images: [Array]

    }

    . . . 

]    

And that's it! You have successfully scraped Google Maps reviews using our API and you can now use the obtained data for various purposes such as data analysis, business analysis, machine learning and more. For further reference and code samples in other 6 programming languages, you can check out our Google Maps reviews documentation.

Limitations of Google Maps Reviews

Even though using a professional scraper to extract Google Maps reviews can be more efficient and accurate than building your own scraper, there are still some limitations to keep in mind. One limitation is that some professional scrapers may have usage limits, which means that you can only scrape a certain number of reviews per day or per month. Another limitation is that some professional scrapers may not be able to bypass IP blocks or CAPTCHAs, which can make it difficult to extract large amounts of data without encountering errors. Luckily, at WebScrapingAPI we have residential proxies which rotate the IP addresses, thus getting you covered and eliminating the worry of being banned or rate limited. One thing you should keep in mind is that Google Maps reviews are usually in natural language, which can make them difficult to analyze and interpret without the use of natural language processing techniques.

Conclusión

In conclusion, scraping Google Maps reviews can be a valuable tool for businesses, researchers, and individuals. It allows you to gather data on a large scale and analyze it for various purposes. However, it's important to keep in mind that there are limitations to scraping Google Maps reviews, including usage limits, CAPTCHAs and IP blocks and natural language processing. Using a professional scraper can make the process more efficient and accurate and can get you rid of some of the limitations.Overall, scraping Google Maps reviews can provide useful information, but it's important to approach it with caution and care.