Intelli-Browser is an open-source Node.js library that allows you to control your browser using natural language, inspired by Claude’s Computer Use. It’s designed for developers, QA professionals, and anyone who spends time navigating the web.

Powered by advanced LLMs (currently only claude-3-5-sonnet) and the Playwright framework, Intelli-Browser helps you automate browser-based tasks, write end-to-end (E2E) tests, and explore the web more efficiently. This tool can transform the way you interact with your browser, tackling repetitive workflows and creating robust test suites.

Features

• Natural Language Commands: Instead of coding intricate test steps, you can direct the browser with simple instructions like “Click search and input ‘Web API’, select the second result, and press ENTER.”
• Traditional e2e Test Case Generation: After executing commands, Intelli-Browser can convert them into structured e2e test cases, which allows you to save and reuse these steps in other tests.
• Customizable API and Logging Options: Intelli-Browser enables you to control logging settings and decide whether to retain context across different sessions, ideal for minimizing API token usage.

How It Works

Intelli-Browser uses the Claude-3.5-Sonnet combined with Playwright to analyze and execute browser actions:

1. User Command: The user types a prompt describing an action (e.g., “Click search and input ‘Web API'”).
2. AI Analysis: The prompt and page details are sent to the LLM, which interprets the page elements and plots the required actions.
3. Action Execution: Intelli-Browser performs the actions as planned by the LLM, updating users on each step.
4. Feedback Loop: If the tool can’t reach a goal or there are no further actions, it halts.

Use Cases

• Automated Website Testing: Quickly create and execute test cases without writing complex code. For example, instruct Intelli-Browser to “Fill out the login form with the test credentials and click submit.”
• Web Scraping and Data Extraction: Specify the data you need in plain English, and let Intelli-Browser handle the navigation and extraction. One example is instructing Intelli-Browser to “Go to the product page and extract the product name, price, and description.”
• Task Automation: Automate repetitive browser tasks. For instance, “Open Gmail, compose a new email to John Doe, and attach the latest sales report.”

Usage Guide

Requirements

To get started, ensure you have Node.js version 18 or above installed. Next, follow the setup steps below to configure Intelli-Browser:

Installation

You can install Intelli-Browser using your preferred package manager:

# npm
npm install @intelli-browser/core
# yarn
yarn add @intelli-browser/core
# pnpm
pnpm add @intelli-browser/core

Setting Up and Running a Demo

Once installed, set up Intelli-Browser by following these steps:

1. Clone the Repository:

   git clone https://github.com/lvqq/intelli-browser.git

2. Install Dependencies:

   pnpm i

3. Build Packages:

   pnpm -r build

4. Add Your API Key: Store your ANTHROPIC_API_KEY in a .env file.
5. Run the Demo:

   pnpm run demo

For direct usage, import Intelli-Browser in your project and start using its powerful natural language capabilities to automate e2e tests with straightforward commands.

Example Usage

The following example shows how to execute a search using Intelli-Browser:

import { IntelliBrowser } from '@intelli-browser/core';
const client = new IntelliBrowser({
  apiKey: '',  // add API key or set ANTHROPIC_API_KEY in .env file
});
await client.run({
  page,  // Pass the Playwright Page instance
  message: 'Click search and input "Web API", press "arrow down" to select the second result, then press "ENTER" to search it',
});

This command directs the browser to search for “Web API” and select the desired result in one seamless action.

Generating Traditional e2e Test Cases

If you’d like to create reusable e2e test cases, Intelli-Browser can generate them automatically after execution. Here’s an example:

const e2e = await client.run({
  page,
  message: 'Click search and input "Web API", press "arrow down" to select the second result, then press "ENTER" to search it',
});
console.log(e2e);  // Logs an array of test steps in code

This output allows users to repurpose test steps for standard automation frameworks, adding flexibility for broader application.

Additional Options

• Disable Logging: Set verbose: false to run actions quietly.
• Retain Context: Use autoClean: false to keep context between actions.

Pros

• Simplified Browser Automation: Control complex browser interactions with ease.
• Reduced Coding Effort: Automate tests and tasks without extensive scripting.
• Improved Efficiency: Accelerate testing and automation workflows.
• Open-Source Flexibility: Customize and extend the library as needed.

Cons

• Limited LLM Support: Currently only supports Claude-3.5-Sonnet.
• Playwright Dependency: Requires the Playwright framework.

Try It Now

Explore Intelli-Browser on GitHub and experience the future of browser automation. Share your thoughts and experiences in the comments below!

Related Resources

• Playwright Documentation: Learn more about the Playwright framework, which powers Intelli-Browser.
• Anthropic API Documentation: Explore the Anthropic API and understand the costs associated with using their language models.