Proxies for Puppeteer

Before you start

You need an active plan, your gateway credentials from the dashboard, and either your server IP whitelisted or a user and password pair. Replace USER:PASS in the examples with your real credentials.

The rotating port gives a new IP on every request. Sticky ports hold one address for the life of the session, and both live on the same host, so you choose by port rather than by product.

The gotcha worth knowing

Puppeteer sends credentials in response to a 407 challenge, which is why they go on the page rather than in the launch flag. Chrome ignores user and password embedded in the proxy-server URL.

Configuration

Puppeteer takes the proxy as a Chrome launch flag, then authenticates per page. The flag has to be set at launch because Chrome reads it once when the browser process starts.

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    args: ['--proxy-server=http://gateway.proxyrotator.com:11000']
  });
  const page = await browser.newPage();

  // Chrome cannot take credentials in the flag, so they go here
  await page.authenticate({ username: 'USER', password: 'PASS' });

  await page.goto('https://api.ipify.org');
  console.log(await page.content());
  await browser.close();
})();

Each new browser launch draws a fresh IP from the rotating pool. If you need one address held across several page loads, use a sticky port instead and keep the same browser instance open.

Where to go next

Running more than one tool against the same gateway is normal. These cover the others.

• All integration guides, setup notes for every tool we cover
• Guides and tutorials, the concepts behind the config, including when to rotate and when not to
• Python requests setup, another library or framework wired to the same gateway
• cURL setup, another library or framework wired to the same gateway
• Rotating proxies, a new IP per request, which is the default for crawling