diff --git a/browsers/pools/overview.mdx b/browsers/pools/overview.mdx index 6ebcbf6..8571482 100644 --- a/browsers/pools/overview.mdx +++ b/browsers/pools/overview.mdx @@ -1,11 +1,15 @@ --- title: "Overview" -description: "Pre-configure pools of reserved browsers" +description: "Pre-configure pools of warm, ready-to-use browsers" --- -Browser pools let you maintain a set of reserved, identical browsers ready for immediate use. Use them to set your preferred browser configuration in advance (such as stealth, proxies, extensions, and profiles), allowing you to minimize browser start-up latency and scale your workloads in production. +Browser pools let you maintain a set of identically-configured browsers ready for immediate use. Use them to set your preferred browser configuration in advance (such as stealth, proxies, extensions, and profiles), allowing you to minimize browser start-up latency and scale your workloads in production. -Acquiring a browser from a pool is faster than creating a browser directly. Reserved browsers and on-demand browsers share the same concurrency limit, and reserved browsers aren't billed until they're used. See [Scale](/introduction/scale) for how pools fit into best practices for production architecture. +Acquiring a browser from a pool is faster than creating a browser directly. Pooled browsers and on-demand browsers share the same concurrency limit, and browsers sitting idle in a pool aren't billed until they're used. See [Scale](/introduction/scale) for how pools fit into best practices for production architecture. + + +New to pools? The [Quickstart](/browsers/pools/quickstart) walks through creating a pool and acquiring your first browser from it. This page is the full API surface. + ## How browser pools work @@ -38,7 +42,7 @@ Browser pools are a way to pre-configure a fixed set of browsers without being c -## Create a pool of reserved browsers +## Create a pool Create a browser pool with a specified size and configuration. All browsers in the pool share the same settings. @@ -268,7 +272,7 @@ You have three ways to get an in-use browser onto the new configuration: ## Per-user profiles with pools -A profile attached to the pool config is [read-only](#create-a-pool-of-reserved-browsers) and shared by every browser, so it can't hold per-user login state across many users in a single pool. To use a pool's pre-warmed browsers to load many different per-user profiles **and** persist each user's state, attach the profile to the browser *after* you acquire it, then destroy the browser on release: +A profile attached to the pool config is [read-only](#create-a-pool) and shared by every browser, so it can't hold per-user login state across many users in a single pool. To use a pool's pre-warmed browsers to load many different per-user profiles **and** persist each user's state, attach the profile to the browser *after* you acquire it, then destroy the browser on release: 1. **Create the pool with no profile.** A profile can only be loaded into a browser that was created without one, so the pool must be profile-free. 2. **Acquire a browser** from the pool. diff --git a/browsers/pools/policy-json.mdx b/browsers/pools/policy-json.mdx index 1332c80..705dc67 100644 --- a/browsers/pools/policy-json.mdx +++ b/browsers/pools/policy-json.mdx @@ -1,13 +1,13 @@ --- title: "Custom Chrome Policies" -description: "Customize Chrome behavior in reserved browser pools using Chrome policies" +description: "Customize Chrome behavior in browser pools using Chrome policies" --- Browser pools accept an optional [`chrome_policy`](https://kernel.sh/docs/api-reference/browser-pools/create-a-browser-pool#body-chrome-policy) object that lets you apply [Chrome enterprise policies](https://chromeenterprise.google/policies/) to every browser in the pool. Use this to control startup behavior, default homepages, bookmarks, and other browser-level settings. ## Setting chrome policies -Pass a `chrome_policy` object when [creating](/browsers/pools/overview#create-a-pool-of-reserved-browsers) or [updating](/browsers/pools/overview#update-a-pool) a pool. Keys are Chrome policy names and values are the corresponding settings. +Pass a `chrome_policy` object when [creating](/browsers/pools/overview#create-a-pool) or [updating](/browsers/pools/overview#update-a-pool) a pool. Keys are Chrome policy names and values are the corresponding settings. ```typescript Typescript/Javascript @@ -176,7 +176,7 @@ The example above demonstrates setting a default homepage and managed bookmarks. ## Common use cases -`chrome_policy` is also accepted directly on `browsers.create()` when you don't need a reserved pool. The examples below use that path. +`chrome_policy` is also accepted directly on `browsers.create()` when you don't need a pool. The examples below use that path. ### Allow pop-ups diff --git a/browsers/pools/quickstart.mdx b/browsers/pools/quickstart.mdx new file mode 100644 index 0000000..f742c39 --- /dev/null +++ b/browsers/pools/quickstart.mdx @@ -0,0 +1,280 @@ +--- +title: "Browser Pools Quickstart" +sidebarTitle: "Quickstart" +description: "Create a pool of pre-warmed browsers and acquire your first one" +--- + +A browser pool keeps a set of identically-configured browsers warm and ready to use. Acquiring a browser from a pool is faster than creating one on demand because the browser has already booted with your stealth, proxy, extension, viewport, and profile settings applied. + +**Idle browsers in a pool aren't billed.** You pay the standard per-second rate only while a browser is acquired and running, so a warm pool costs nothing between tasks. Pool capacity does count against your [concurrency limit](/info/pricing#concurrency-limits) — a pool of 20 uses 20 of your limit whether or not those browsers are acquired. + + + Browser pools are available on the Start-Up and Enterprise plans. Install the Kernel SDK first: + - Typescript/Javascript: `npm install @onkernel/sdk` + - Python: `pip install kernel` + - Go: `go get github.com/kernel/kernel-go-sdk` + + +## Create the pool + +Create the pool once — at deploy time, on service startup, or by hand from the CLI or [dashboard](https://dashboard.onkernel.com). Keep it out of the code path that serves your workload: pools fill at 25% of their size per minute by default, so a pool of 20 takes about four minutes to warm up completely. + + +```typescript Typescript/Javascript +import Kernel from '@onkernel/sdk'; + +const kernel = new Kernel(); + +const pool = await kernel.browserPools.create({ + name: 'checkout-pool', + size: 20, + stealth: true, + timeout_seconds: 600, +}); + +console.log(pool.id); +``` + +```python Python +from kernel import Kernel + +kernel = Kernel() + +pool = kernel.browser_pools.create( + name="checkout-pool", + size=20, + stealth=True, + timeout_seconds=600, +) + +print(pool.id) +``` + +```go Go +package main + +import ( + "context" + "fmt" + + "github.com/kernel/kernel-go-sdk" +) + +func main() { + ctx := context.Background() + client := kernel.NewClient() + + pool, err := client.BrowserPools.New(ctx, kernel.BrowserPoolNewParams{ + Name: kernel.String("checkout-pool"), + Size: 20, + Stealth: kernel.Bool(true), + TimeoutSeconds: kernel.Int(600), + }) + if err != nil { + panic(err) + } + + fmt.Println(pool.ID) +} +``` + +```bash CLI +kernel browser-pools create checkout-pool --size 20 --stealth --timeout 600 +``` + + +Every browser in the pool gets the same configuration, so put whatever your workload needs here — proxies, extensions, a viewport, a profile, custom [Chrome policies](/browsers/pools/policy-json). See [Pool configuration options](/browsers/pools/overview#pool-configuration-options) for the full list. + +## Acquire, drive, and release a browser + +Now do the work. `acquire` returns a browser immediately if one is available, or waits until one is. The response carries the same fields as an on-demand browser — `session_id`, `cdp_ws_url`, `browser_live_view_url` — so your automation code doesn't change. + +Release the browser when you're done. Release in a `finally` block: a browser you don't release stays acquired until its idle timeout expires, which shrinks the pool everyone else is acquiring from. + + +```typescript Typescript/Javascript +import Kernel from '@onkernel/sdk'; + +const kernel = new Kernel(); + +const kernelBrowser = await kernel.browserPools.acquire('checkout-pool', { + acquire_timeout_seconds: 30, +}); + +try { + const response = await kernel.browsers.playwright.execute( + kernelBrowser.session_id, + { + code: ` + await page.goto('https://www.onkernel.com'); + return await page.title(); + `, + }, + ); + console.log(response.result); +} finally { + await kernel.browserPools.release('checkout-pool', { + session_id: kernelBrowser.session_id, + }); +} +``` + +```python Python +from kernel import Kernel + +kernel = Kernel() + +kernel_browser = kernel.browser_pools.acquire( + "checkout-pool", + acquire_timeout_seconds=30, +) + +try: + response = kernel.browsers.playwright.execute( + id=kernel_browser.session_id, + code=""" + await page.goto('https://www.onkernel.com') + return await page.title() + """, + ) + print(response.result) +finally: + kernel.browser_pools.release( + "checkout-pool", + session_id=kernel_browser.session_id, + ) +``` + +```go Go +package main + +import ( + "context" + "fmt" + + "github.com/kernel/kernel-go-sdk" +) + +func main() { + ctx := context.Background() + client := kernel.NewClient() + + kernelBrowser, err := client.BrowserPools.Acquire(ctx, "checkout-pool", kernel.BrowserPoolAcquireParams{ + AcquireTimeoutSeconds: kernel.Int(30), + }) + if err != nil { + panic(err) + } + defer func() { + if err := client.BrowserPools.Release(ctx, "checkout-pool", kernel.BrowserPoolReleaseParams{ + SessionID: kernelBrowser.SessionID, + }); err != nil { + panic(err) + } + }() + + response, err := client.Browsers.Playwright.Execute(ctx, kernelBrowser.SessionID, kernel.BrowserPlaywrightExecuteParams{ + Code: ` + await page.goto('https://www.onkernel.com'); + return await page.title(); + `, + }) + if err != nil { + panic(err) + } + + fmt.Println(response.Result) +} +``` + +```bash CLI +kernel browser-pools acquire checkout-pool --timeout 30 + +# then, when you're done with the session it returned +kernel browser-pools release checkout-pool --session-id +``` + + +By default, released browsers are reused: the browser goes back into the pool as-is and the next `acquire` gets it. Pass `reuse: false` to destroy it instead and have the pool warm a replacement — use that when the browser holds state you don't want the next task to see. See [Release a browser](/browsers/pools/overview#release-a-browser). + +## Check on the pool + +`available_count` tells you how many browsers are ready to be acquired right now, and `acquired_count` how many are in use. + + +```typescript Typescript/Javascript +const pool = await kernel.browserPools.retrieve('checkout-pool'); + +console.log(pool.available_count, pool.acquired_count); +``` + +```python Python +pool = kernel.browser_pools.retrieve("checkout-pool") + +print(pool.available_count, pool.acquired_count) +``` + +```go Go +pool, err := client.BrowserPools.Get(ctx, "checkout-pool") +if err != nil { + panic(err) +} + +fmt.Println(pool.AvailableCount, pool.AcquiredCount) +``` + +```bash CLI +kernel browser-pools get checkout-pool +``` + + +Watch this number to size the pool. If `available_count` regularly hits 0, your tasks are queueing behind `acquire` and the pool needs to be bigger. If it stays above 30–40% of the pool size under normal load, you can shrink the pool and free up concurrency for other work. Target 10–20% available during typical load. + +Resize at any time with `update`; there's no need to tear the pool down and rebuild it. Added capacity warms up at the pool's fill rate, so raise the size before a traffic peak rather than during it: + + +```typescript Typescript/Javascript +await kernel.browserPools.update('checkout-pool', { size: 40 }); +``` + +```python Python +kernel.browser_pools.update("checkout-pool", size=40) +``` + +```go Go +if _, err := client.BrowserPools.Update(ctx, "checkout-pool", kernel.BrowserPoolUpdateParams{ + Size: 40, +}); err != nil { + panic(err) +} +``` + +```bash CLI +kernel browser-pools update checkout-pool --size 40 +``` + + +## Before you go to production + +Three things behave differently from on-demand browsers: + +- **A pool's `timeout_seconds` only runs while a browser is acquired.** Browsers wait in the pool indefinitely; the clock starts when you acquire one and only counts idle time — no CDP connection, no live view, no in-flight computer controls request. A task that runs for 20 minutes with an active CDP connection won't be killed by a 10-minute timeout. See [Timeout behavior](/browsers/pools/overview#timeout-behavior). +- **Updating a pool doesn't reconfigure browsers that already exist.** `update` changes the configuration used for browsers created after it. Pass `discard_all_idle: true` to rebuild the idle ones too; acquired browsers keep their old configuration even then. See [Update a pool](/browsers/pools/overview#update-a-pool). +- **A profile set on the pool is read-only.** Every browser in the pool shares it, so `save_changes` doesn't apply at the pool level. To give each of your users their own persisted login state, attach the profile after you acquire the browser — see [Per-user profiles with pools](/browsers/pools/overview#per-user-profiles-with-pools). + +## What's next + + + + The full pool API — configuration options, profile refresh, flushing, and deletion. + + + Architecture patterns for high concurrency, including pools behind a task queue. + + + Apply Chrome enterprise policies — pop-up handling, download behavior, homepages — to every browser in a pool. + + + Sizing, fill rate, reuse semantics, and debugging pooled browsers in production. + + diff --git a/docs.json b/docs.json index 9c781c5..6a217ba 100644 --- a/docs.json +++ b/docs.json @@ -89,6 +89,15 @@ "browsers/termination", "browsers/standby", "browsers/headless", + { + "group": "Browser Pools", + "pages": [ + "browsers/pools/quickstart", + "browsers/pools/overview", + "browsers/pools/policy-json", + "browsers/pools/faq" + ] + }, "info/projects" ] }, @@ -160,14 +169,6 @@ "browsers/telemetry/categories", "browsers/telemetry/streaming" ] - }, - { - "group": "Reserved Browsers", - "pages": [ - "browsers/pools/overview", - "browsers/pools/policy-json", - "browsers/pools/faq" - ] } ] }, diff --git a/index.mdx b/index.mdx index d610467..36f4cfa 100644 --- a/index.mdx +++ b/index.mdx @@ -75,4 +75,4 @@ kernel invoke my-agent my-task --payload '{"url": "https://example.com"}' ### scaling -once you're ready to scale, check out how to create a [pool of reserved browsers](/browsers/pools/overview). \ No newline at end of file +[browser pools](/browsers/pools/quickstart) keep browsers warm and pre-configured so you skip start-up latency on every task. idle pooled browsers aren't billed, so most production workloads run on them from day one. \ No newline at end of file diff --git a/info/concepts.mdx b/info/concepts.mdx index a814b72..0061e51 100644 --- a/info/concepts.mdx +++ b/info/concepts.mdx @@ -5,6 +5,8 @@ title: "Concepts" ## Browser A `Browser` is a cloud-based browser managed by Kernel. They accept Chrome DevTools Protocol connections and can be used to run browser automations or web agents. +## Browser Pool +A `Browser Pool` is a set of identically-configured browsers that Kernel keeps warm and ready to use. You acquire a browser from the pool when a task starts and release it back when the task finishes. Pools remove browser start-up latency from your workload — see [Browser Pools](/browsers/pools/quickstart). ## App An `App` is a codebase deployed on Kernel. You can use Kernel for a variety of use cases, including web automations, data processing, and more. diff --git a/info/pricing.mdx b/info/pricing.mdx index 08bc4db..1b287c3 100644 --- a/info/pricing.mdx +++ b/info/pricing.mdx @@ -45,9 +45,9 @@ import { PricingCalculator } from '/snippets/calculator.jsx'; > Note: Included monthly credits apply to usage costs only. -## Reserved browsers (browser pools) +## Browser pools -With Browser Pools, you pay the standard usage-based price per GB-second while browsers are running. Idle browsers in a pool incur no disk charges — you only pay when a browser is actively in use. +With [browser pools](/browsers/pools/quickstart), you pay the standard usage-based price per GB-second while browsers are running. Idle browsers in a pool incur no disk charges — you only pay when a browser is actively in use, so keeping a pool warm between tasks costs nothing. > Note: GPU acceleration is not available for browser pools. @@ -59,7 +59,7 @@ Auth sessions are fast (typically 5-30 seconds each). Kernel monitors session he ## Concurrency limits -Kernel enforces a single concurrency limit covering all browsers you run at once — whether created on demand with `browsers.create()` or reserved in a [browser pool](/browsers/pools/overview). Your full limit is available to either API in any mix. +Kernel enforces a single concurrency limit covering all browsers you run at once — whether created on demand with `browsers.create()` or warmed in a [browser pool](/browsers/pools/overview). Your full limit is available to either API in any mix. | Feature | Developer (free + usage) | Hobbyist ($30 / mo + usage) | Start-Up ($200 / mo + usage) | Enterprise | | --- | --- | --- | --- | --- | @@ -68,7 +68,7 @@ Kernel enforces a single concurrency limit covering all browsers you run at once | App invocations (per-app) | 5 | 10 | 20 | Custom | | Managed auth health check interval | N/A | 1 hour minimum | 20 minutes minimum | Custom | -> Note: Reserved capacity in a [browser pool](/browsers/pools/overview) counts toward your concurrency limit whether or not the browsers are currently acquired — a pool sized to 40 browsers uses 40 of your limit. Browser pools are available on Start-Up and Enterprise plans. +> Note: A [browser pool](/browsers/pools/overview) counts toward your concurrency limit whether or not its browsers are currently acquired — a pool sized to 40 browsers uses 40 of your limit. Browser pools are available on Start-Up and Enterprise plans. > Note: Browsers in [Standby Mode](/browsers/standby) count against your concurrency limit. diff --git a/info/projects.mdx b/info/projects.mdx index ed08360..222db9a 100644 --- a/info/projects.mdx +++ b/info/projects.mdx @@ -320,9 +320,9 @@ if err := client.Projects.Delete(ctx, "proj_abc123"); err != nil { ## Concurrency Limits -Kernel caps how many browsers can run at once, at two levels. A single limit covers both on-demand browsers (`browsers.create()`) and [browser pools](/browsers/pools/overview) — standalone sessions and reserved pool capacity count against the same cap. +Kernel caps how many browsers can run at once, at two levels. A single limit covers both on-demand browsers (`browsers.create()`) and [browser pools](/browsers/pools/overview) — standalone sessions and pool capacity count against the same cap. -- **Organization limit** — the total concurrent browsers allowed across your whole organization, determined by your plan. Every browser session and every reserved pool slot counts against it. +- **Organization limit** — the total concurrent browsers allowed across your whole organization, determined by your plan. Every browser session and every browser in a pool counts against it. - **Per-project limits** — optional caps on individual projects, so one team or environment can't consume the entire org limit. Per-project caps come from two places: diff --git a/introduction/create.mdx b/introduction/create.mdx index beaf91c..2c3ba40 100644 --- a/introduction/create.mdx +++ b/introduction/create.mdx @@ -85,6 +85,54 @@ Most of what you'll tune at creation time falls into four buckets: +## On demand or from a pool + +`browsers.create()` boots a browser for you on the spot. That's the right call while you're building, and for workloads that run occasionally. + +Once you're running the same task repeatedly — or more than a handful at a time — create a [browser pool](/browsers/pools/quickstart) instead. A pool holds browsers that are already booted with your configuration applied, so `acquire` hands you one that's ready to drive rather than starting one from scratch. Two things get faster: configurations that restart Chromium on creation (custom viewports, extensions, kiosk mode) are already applied, and acquiring from a pool sidesteps the [rate limit](/info/pricing#rate-limiting) on browser creation that you'll otherwise hit at volume. + + +```typescript Typescript/Javascript +// Create the pool once, at deploy time or on startup +await kernel.browserPools.create({ name: 'checkout-pool', size: 20, stealth: true }); + +// Then, wherever your automation runs +const kernelBrowser = await kernel.browserPools.acquire('checkout-pool', {}); +``` + +```python Python +# Create the pool once, at deploy time or on startup +kernel.browser_pools.create(name="checkout-pool", size=20, stealth=True) + +# Then, wherever your automation runs +kernel_browser = kernel.browser_pools.acquire("checkout-pool") +``` + +```go Go +// Create the pool once, at deploy time or on startup +if _, err := client.BrowserPools.New(ctx, kernel.BrowserPoolNewParams{ + Name: kernel.String("checkout-pool"), + Size: 20, + Stealth: kernel.Bool(true), +}); err != nil { + panic(err) +} + +// Then, wherever your automation runs +kernelBrowser, err := client.BrowserPools.Acquire(ctx, "checkout-pool", kernel.BrowserPoolAcquireParams{}) +if err != nil { + panic(err) +} +``` + +```bash CLI +kernel browser-pools create checkout-pool --size 20 --stealth +kernel browser-pools acquire checkout-pool +``` + + +An acquired browser returns the same fields as one you created directly, so the rest of your code is identical. Idle browsers in a pool aren't billed — you pay only while a browser is acquired and running — though pool capacity does count against your [concurrency limit](/info/pricing#concurrency-limits). + ## Lifecycle A browser stays alive as long as something is driving it — a CDP or WebDriver client, a [Live View](/browsers/live-view) viewer, or an in-flight [computer controls](/browsers/computer-controls) request. After five seconds with none of those active, it enters [standby](/browsers/standby) — state is preserved, billing stops. Once in standby, after the configurable timeout (60s by default) elapses it's deleted. @@ -211,3 +259,5 @@ func main() { ## What's next Once you have a browser, you need to drive it. Head to [Control](/introduction/control) to see the four primitives Kernel exposes — computer use, playwright execution, CDP, and WebDriver BiDi — and when to reach for each. + +When you're ready to run this in production, the [Browser Pools Quickstart](/browsers/pools/quickstart) shows how to serve your workload from warm, pre-configured browsers instead of creating one per task. diff --git a/introduction/scale.mdx b/introduction/scale.mdx index ae25500..746c71b 100644 --- a/introduction/scale.mdx +++ b/introduction/scale.mdx @@ -8,6 +8,8 @@ This guide explains how to architect production-scale browser automation systems After understanding the basics of [creating](/introduction/create) and [controlling](/introduction/control) our browsers, you should understand how to create and connect to individual browsers on-demand. This guide builds on that foundation to help you design systems using browser pools that can handle hundreds or thousands of concurrent browser tasks reliably. +For the mechanics of standing up a pool and acquiring browsers from it, start with the [Browser Pools Quickstart](/browsers/pools/quickstart). This guide covers which architecture to wrap around it. + ## Understanding your requirements Before selecting an architecture, assess your workload's characteristics: