LinkedIn company pages are among the most scraped targets on the web, and for good reason. Recruiters, sales teams, and market researchers all need fresh data on company size, industry, and contact details. This guide walks you through building a working LinkedIn company scraper using Crawlee and Playwright, packaged as an Apify actor you can run on a schedule or sell on the Apify Store.
You'll learn how to structure the actor, handle LinkedIn's anti-bot systems, extract the key fields, and publish the final package.
A note on terms of service: LinkedIn's ToS restricts automated access to profile and company data. This guide focuses on publicly visible company pages and rate-limited, respectful crawling. Always review applicable terms and local data-protection laws before running scrapers in production.
What Data Can a LinkedIn Company Scraper Extract?
Public LinkedIn company pages expose a reliable set of structured fields without requiring login for most of them. A well-built linkedin company scraper can pull:
- Company name - the canonical name as listed on the page
- Industry - the category LinkedIn assigns (e.g., "Software Development")
- Employee count - the headcount range LinkedIn displays, such as "1,001-5,000 employees"
- Website - the external URL listed in the company's About section
- Description - the "About us" text, typically 100-500 words
- Headquarters - city, state, and country
- Founded year - when the company was established
- Specialties - comma-separated list of focus areas
Some fields (employee count, specialties) only render after JavaScript executes, which is why Playwright is a better fit here than a simple HTTP client like axios.
How to Structure an Apify Actor for LinkedIn Data
An Apify actor is a Node.js application with a specific entry point and a standard file layout. Before writing any scraping logic, get the scaffold right.
Required File Layout
my-linkedin-scraper/
src/
main.js # entry point
package.json
actor.json # actor metadata
.actor/
actor.json # Apify actor config (alternative location)
package.json
LinkedIn pages are JavaScript-heavy, so Playwright is the right tool. Crawlee's PlaywrightCrawler handles request queuing, retry logic, and browser lifecycle automatically. The Apify actor SDK docs cover the full Actor API if you need to go deeper on input schema or dataset handling.
{
"name": "linkedin-company-scraper",
"version": "1.0.0",
"type": "module",
"description": "Scrapes public LinkedIn company pages",
"main": "src/main.js",
"scripts": {
"start": "node src/main.js"
},
"dependencies": {
"apify": "^3.2.0",
"crawlee": "^3.9.0",
"playwright": "^1.44.0",
"camoufox-js": "^0.1.0"
}
}
The "type": "module" field is required. Apify v3 is ESM-first, and mixing CommonJS require() calls with ESM packages causes cryptic errors at runtime.
actor.json
{
"actorSpecification": 1,
"name": "linkedin-company-scraper",
"title": "LinkedIn Company Scraper",
"description": "Extracts company name, industry, size, website, and description from public LinkedIn pages.",
"version": "0.1",
"input": {
"schemaVersion": 1,
"fields": {
"companyUrls": {
"title": "Company URLs",
"type": "array",
"description": "List of LinkedIn company page URLs to scrape.",
"editor": "stringList"
},
"maxRequestsPerCrawl": {
"title": "Max requests",
"type": "integer",
"default": 50
}
}
}
}
How Does LinkedIn Detect and Block Scrapers?
LinkedIn runs several layers of bot detection. Understanding each one helps you build a linkedin company scraper that actually completes runs instead of getting blocked on the first page.
Fingerprinting: LinkedIn's front-end JavaScript inspects browser properties - navigator.webdriver, canvas fingerprints, missing browser APIs - that headless Chromium exposes by default. A vanilla Playwright launch fails these checks quickly.
Rate limiting: Too many requests from a single IP in a short window triggers a CAPTCHA or a 999 response code (LinkedIn's custom "bot detected" status).
Cookie and session checks: Requests without a plausible cookie jar get redirected to the login page after the first page or two, even for data that's technically public.
Using Camoufox for Stealth Browsing
Camoufox is a patched Firefox build designed to pass browser fingerprint checks. The camoufox-js npm package exposes a launchOptions helper that generates the correct Firefox flags, and you pair it with firefox from Playwright. It randomizes canvas fingerprints, fixes navigator.webdriver, and spoofs hundreds of browser properties that bots typically expose.
// src/main.js
import { Actor } from 'apify';
import { PlaywrightCrawler } from 'crawlee';
import { launchOptions as camoufoxLaunchOptions } from 'camoufox-js';
import { firefox } from 'playwright';
await Actor.init();
const input = await Actor.getInput();
const { companyUrls = [], maxRequestsPerCrawl = 50 } = input ?? {};
const proxyConfiguration = await Actor.createProxyConfiguration();
const crawler = new PlaywrightCrawler({
maxRequestsPerCrawl,
proxyConfiguration,
launchContext: {
launcher: firefox, // use Firefox instead of Chromium
launchOptions: await camoufoxLaunchOptions({
headless: true,
geoip: true, // match IP geolocation to browser locale
}),
},
async requestHandler({ page, request, log }) {
log.info(`Scraping: ${request.url}`);
// Wait for the about section to be present
await page.waitForSelector('section.org-page-details', { timeout: 15000 });
const data = await page.evaluate(() => {
const getText = (selector) =>
document.querySelector(selector)?.innerText?.trim() ?? null;
return {
name: getText('h1.org-top-card-summary__title'),
industry: getText('.org-top-card-summary-info-list__info-item:nth-child(1)'),
employeeCount: getText('.org-top-card-summary-info-list__info-item:nth-child(2)'),
headquarters: getText('.org-top-card-summary-info-list__info-item:nth-child(3)'),
website: document.querySelector('a[data-field="website"]')?.href ?? null,
description: getText('.org-page-details__definition-text'),
founded: getText('.org-page-details__definition-text + dd'),
specialties: getText('.org-page-details__specialities'),
};
});
await Actor.pushData({ url: request.url, ...data });
},
failedRequestHandler({ request, log }) {
log.error(`Failed: ${request.url}`);
},
});
await crawler.run(companyUrls);
await Actor.exit();
A few things worth calling out in this code:
launcher: firefoxreplaces Playwright's default Chromium with Firefox, which Camoufox patches. The rest of thePlaywrightCrawlerAPI stays identical.camoufoxLaunchOptionsis async - it generates browser flags at runtime, so you needawaitbefore it.waitForSelectoronsection.org-page-detailsensures the JavaScript-rendered content has loaded before scraping. Skipping this step produces mostly empty fields.Actor.pushData()writes each record to Apify's default dataset, which becomes the actor's output.
Adding Request Delays
Camoufox handles fingerprinting, but you still need to throttle requests. PlaywrightCrawler accepts a minConcurrency / maxConcurrency pair and a navigationTimeoutSecs option, but for deliberate pacing, a delay inside the handler works cleanly:
// Inside requestHandler, before pushData
await page.waitForTimeout(2000 + Math.random() * 3000); // 2-5s random delay
Random delays are more effective than fixed ones because fixed intervals are detectable as a pattern.
How Do You Handle LinkedIn Login Walls?
Some company data (full employee lists, certain headcount breakdowns) only appears after login. For publicly visible fields, the fields listed above are accessible without authentication on most company pages. If you find the page redirecting to /login, two approaches work:
Approach 1 - Pass a Session Cookie
If you have a valid LinkedIn session, you can export your cookies from a browser and inject them into Playwright:
launchContext: {
launcher: firefox,
launchOptions: {
...(await camoufoxLaunchOptions({ headless: true })),
userDataDir: './browser-profile', // persist cookies across runs
},
},
Then log in manually on the first run and let the browser profile carry the session for subsequent runs. This keeps the session alive as long as LinkedIn doesn't invalidate it.
Approach 2 - Limit to Public Data
Restrict your scraper to the subset of fields available without login. The company name, industry, description, website, and employee range are reliably available on public pages. This is the safer long-term approach for a public Apify actor, since you won't be distributing credentials.
How Do You Publish a LinkedIn Company Scraper to the Apify Store?
Once your actor runs cleanly locally with apify run, publishing takes three steps.
Step 1 - Connect Your Apify Account
In your Coznix dashboard, go to Settings and enter your Apify API token. Coznix uses this token to push actors directly to your Apify account. You can also find ready-to-customize LinkedIn data actors on the Apify Store by searching with Coznix - it scans demand scores and scrapability ratings so you can pick niches where competition is low.
Step 2 - Push the Actor via Apify CLI
apify push
This command zips your source files and creates or updates the actor in your Apify account. The first push creates the actor; subsequent pushes create a new version.
Step 3 - Set the Build and Run Configuration
In the Apify Console, set the actor's default run memory to at least 1 GB. Playwright with Camoufox needs roughly 700-900 MB per browser instance. Running at 256 MB causes silent crashes with no useful error message.
Set the default timeout to at least 5 minutes per run for batches of 50 or fewer URLs. Longer batches need proportionally more time.
What Are the Most Common Errors When Scraping LinkedIn?
Even with Camoufox in place, a few errors come up regularly.
999 response code: LinkedIn's bot signal. Slow down your request rate and rotate your user agent. If you're running on a shared proxy IP that's already flagged, switch to a residential proxy pool.
waitForSelector timeout: The selector changed or the page didn't load the expected section. Add a fallback: try a second selector if the first fails, and log the page HTML for debugging when both fail.
Empty fields: The evaluate() call ran before JavaScript rendered the section. Increase the timeout on waitForSelector or add an explicit page.waitForLoadState('networkidle') before scraping.
Actor memory exceeded: Reduce maxConcurrency to 1 or 2, or increase the actor's memory limit in the Apify Console.
Frequently Asked Questions
Can I scrape LinkedIn without Playwright? For most company page fields, yes - Cheerio and a plain HTTP request sometimes work on the initial HTML load. But employee count, specialties, and the description often don't appear in the initial HTML and require JavaScript execution. Playwright covers both cases reliably.
How many company pages can I scrape per hour? With a 2-5 second random delay and a single browser instance, expect around 30-60 pages per hour. More concurrency increases throughput but also ban risk. For large batches, residential proxies and lower concurrency tend to produce better long-term results than pushing speed.
Does this work for LinkedIn personal profiles?
The selectors in this guide target company pages (/company/ URLs). Personal profile pages use different selectors and have stricter bot detection. The same actor structure applies, but you'd need different page.evaluate() selectors.
Where can I find existing LinkedIn scrapers on the Apify Store? Coznix indexes the Apify Store and scores actors by demand and scrapability. Search "linkedin company" to see what's already available and how much demand each actor is getting - useful before you spend time building something that already exists.
Build and Ship Faster
Building a LinkedIn company scraper from scratch teaches you a lot about anti-bot handling and browser automation. The actor structure covered here, Crawlee's PlaywrightCrawler, Camoufox for stealth, and Apify's pushData output pattern, applies to virtually any JavaScript-rendered site you want to scrape next.
If you want to skip straight to finding high-demand, low-competition scraping opportunities and generating production-ready actor code without writing the scaffold yourself, sign up for Coznix. You get 50 free credits to analyze the Apify Store, score opportunities, and generate a working actor in minutes.