DuckDB-Powered In-Browser Analytics

Last update on October 12 2026 11:06:00 (UTC/GMT +8 hours)

DuckDB-Powered In-Browser Analytics : A Browser-Based SQL Playground

For decades, the workflow for analyzing a dataset has followed a predictable pattern: upload the file to a server, wait for it to be processed, and query it remotely. This model works, but it carries inherent costs — infrastructure expenses, latency, and perhaps most significantly, a loss of privacy. When you upload a spreadsheet containing customer records, financial data, or medical information to a third-party server, you are trusting that server with sensitive information. For many organizations and individuals, that trust is not something they are willing to extend.

DuckDB-Powered In-Browser Analytics represents a fundamental inversion of this model. Instead of sending data to the computation, it brings the computation to the data. Users upload CSV or Parquet files directly in the browser, and DuckDB-WASM — a full-featured analytical database compiled to WebAssembly — executes SQL queries entirely client-side. No server round-trips. No data leaving the user's machine. No infrastructure costs. The results render as interactive tables and charts, and queries can be saved and shared as URLs.

This is not a toy or a demonstration. DuckDB is a genuine analytical database, designed for the kind of columnar, vectorized processing that powers modern data warehouses. Running it in the browser, backed by Apache Arrow for zero-copy data interchange, delivers performance that would have seemed impossible for a client-side application just a few years ago.

What It Does

The application's workflow is deliberately simple, removing every barrier between a user and their data.

Upload: Users drag and drop CSV or Parquet files into the browser. No upload progress bar, because there is no upload — the file is read directly from the local filesystem into memory. Files of hundreds of megabytes are feasible on modern hardware; larger files benefit from Parquet's compression.

Query: Once loaded, the data appears as a table available for SQL querying. Users write standard SQL — SELECT, JOIN, GROUP BY, window functions, CTEs — and execute it against their local data. DuckDB's SQL dialect is PostgreSQL-compatible and rich with analytical extensions, so users familiar with any SQL database can be productive immediately.

Visualize: Query results render as interactive tables with sorting, filtering, and pagination. For numerical results, the application suggests and renders charts — bar charts for categorical comparisons, line charts for time series, scatter plots for correlations. The visualization layer is responsive, updating as queries change.

Persist and Share: Queries can be saved locally for later use, or encoded into a URL that captures both the query and a reference to the data source. Sharing a URL lets a colleague reproduce the exact analysis, provided they have access to the same data.

The experience is that of a desktop analytics application — responsive, private, and self-contained — delivered through a web browser.

Architecture and Data Flow

DuckDB-WASM: The Analytical Engine

DuckDB is an in-process analytical database, often described as "SQLite for analytics." Where SQLite optimizes for transactional workloads (many small reads and writes), DuckDB optimizes for analytical workloads (large scans, aggregations, joins). It uses columnar storage, vectorized execution, and aggressive parallelism to process analytical queries far faster than row-oriented databases.

DuckDB-WASM is DuckDB compiled to WebAssembly, allowing it to run in any modern browser at near-native speed. The compilation preserves DuckDB's core architecture: the same query optimizer, the same execution engine, the same SQL dialect. What changes is the runtime environment — instead of linking against system libraries, it runs within the browser's sandbox, accessing files through the File API and storing data in memory or IndexedDB.

The browser context imposes constraints that shape the architecture. Memory is limited compared to a server. Web Workers provide background threads, but the main thread must remain responsive. DuckDB-WASM addresses these constraints by running queries in a Web Worker, keeping the UI thread free, and by streaming results incrementally rather than materializing everything at once.

Loading Data: CSV and Parquet

DuckDB can read CSV and Parquet files directly, inferring schema from the data. For CSV, the reader handles delimiters, quoting, headers, and type inference — recognizing integers, floats, dates, and strings. For Parquet, the reader benefits from the format's embedded schema and columnar layout, enabling projection pushdown (reading only the columns a query needs) and predicate pushdown (skipping row groups that cannot match a filter).

Loading a file in the browser involves reading it as an ArrayBuffer and registering it with DuckDB's virtual filesystem:

javascript:


import * as duckdb from '@duckdb/duckdb-wasm';

const db = await duckdb.createDuckDB();
const conn = await db.connect();

// Register a file in DuckDB's virtual filesystem
await db.registerFileBuffer('data.parquet', new Uint8Array(arrayBuffer));

// Query it directly
const result = await conn.query(`
  SELECT category, SUM(revenue) AS total
  FROM 'data.parquet'
  GROUP BY category
  ORDER BY total DESC
`);

The registerFileBuffer method makes the file available to DuckDB without copying it into a table. Queries reference the file by name, and DuckDB reads it through its virtual filesystem. This lazy approach is efficient: data is read only when needed, and Parquet's columnar format means only relevant columns are touched.

For repeated queries, materializing the data into a DuckDB table can improve performance, since it avoids re-parsing the file on each query:

sql:


CREATE TABLE sales AS SELECT * FROM 'data.parquet';

Tables created this way live in DuckDB's memory (or IndexedDB, if persistence is configured) and support indexing and other optimizations.

Apache Arrow: Zero-Copy Data Interchange

Apache Arrow is a language-agnostic columnar memory format designed for efficient data interchange. It is the connective tissue between DuckDB and the JavaScript visualization layer.

DuckDB-WASM returns query results as Arrow tables, not as JavaScript objects. This is a crucial design decision. Converting a million-row result into a million JavaScript objects would be catastrophically slow — each object carries overhead, and the conversion would dominate query time. Arrow tables, by contrast, store data in contiguous typed arrays that can be passed across the WebAssembly boundary with minimal copying.

The Arrow format is also columnar, which aligns with how charts and tables consume data. A bar chart needs all values from one column, not row-by-row records. Arrow's columnar layout delivers exactly that, enabling efficient iteration and aggregation.

In practice, the flow is:

DuckDB executes the query and produces an Arrow table
The Arrow table is transferred from the Web Worker to the main thread
The visualization layer reads columns directly from the Arrow table
Charts and tables render from Arrow data without intermediate conversion

This pipeline is fast enough that query results appear essentially instantly for typical analytical queries.

Persistence with IndexedDB

While in-memory processing is fast, it is also ephemeral. Reload the page, and the loaded data is gone. For workflows that involve repeated analysis or large files, this is inconvenient.

IndexedDB provides browser-native persistent storage, and DuckDB-WASM can use it as its filesystem backing. Data loaded into IndexedDB tables survives page reloads, allowing users to return to their analysis without re-uploading files. The trade-off is performance: IndexedDB is slower than memory, so the application must decide which tables to persist and which to keep in memory.

A practical approach persists the source data (the uploaded files or materialized tables) in IndexedDB while keeping query results in memory. This balances durability with speed — the expensive-to-recreate data is preserved, while the cheap-to-recompute results are not.

Optional Cloud Sync with FastAPI

The application is fundamentally client-side, but some features benefit from an optional server component. A FastAPI backend can provide:

Query sharing: Storing saved queries and their metadata, enabling sharing across users and devices
Data source registration: Cataloging available datasets without storing the data itself
Collaboration: Comment threads, annotations, or shared workspaces
Authentication: User accounts for managing saved queries and preferences

Critically, the cloud sync is optional. The core analytics functionality — upload, query, visualize — works entirely offline. The server enhances the experience without being required for it. And even when the server is used, it never sees the user's data, only their queries and metadata. This preserves the privacy-first promise that motivates the tool.

Python UDFs with Pyodide

Some analyses require logic that is easier to express in Python than in SQL. Pyodide — a Python runtime compiled to WebAssembly — can execute Python code in the browser, and DuckDB can call into it for user-defined functions.

This enables patterns like:

python:


# Define a Python UDF in the browser
def sentiment_score(text):
    # Simplified sentiment analysis
    positive = sum(word in text.lower() for word in ['good', 'great', 'excellent'])
    negative = sum(word in text.lower() for word in ['bad', 'poor', 'terrible'])
    return positive - negative

# Register it with DuckDB
con.create_function('sentiment', sentiment_score, [str], int)

# Use it in SQL
result = con.execute("""
    SELECT product, sentiment(review_text) AS score
    FROM reviews
    WHERE sentiment(review_text) > 0
""")

Pyodide's integration with DuckDB-WASM is more involved than pure SQL usage — Python startup takes time, and data transfer between JavaScript, WebAssembly, and Python has overhead. But for analyses that genuinely benefit from Python's ecosystem, it opens possibilities that SQL alone cannot reach.

Modern Technology Stack

DuckDB-WASM

The analytical engine at the heart of the application. DuckDB's columnar, vectorized architecture delivers analytical query performance that rivals server-based databases, and the WASM compilation makes it portable to any browser. The project is actively maintained, with regular releases that improve performance and add features.

Apache Arrow

The data interchange format that connects DuckDB to the visualization layer. Arrow's columnar memory layout and zero-copy semantics eliminate the serialization overhead that would otherwise dominate query result handling. The JavaScript Arrow libraries (apache-arrow) integrate cleanly with both DuckDB-WASM and modern charting libraries.

Pyodide

The Python runtime for WebAssembly, enabling Python UDFs. Pyodide brings the scientific Python stack — NumPy, Pandas, scikit-learn — into the browser, though loading these packages has significant startup cost. For applications that need occasional Python logic, it is a powerful addition; for applications that need only SQL, it is unnecessary.

FastAPI

The optional backend for cloud sync, query sharing, and collaboration. FastAPI's async architecture and Pydantic validation make it well-suited to these tasks, and its lightweight footprint keeps the server component minimal.

IndexedDB

The browser's persistent storage mechanism, used to cache uploaded data and materialized tables. IndexedDB's asynchronous API and structured storage make it suitable for large binary data, though its performance characteristics require careful design.

User Benefits

Privacy-First Analytics

The defining benefit of in-browser analytics is that data never leaves the user's machine. For sensitive datasets — medical records, financial data, personal information — this eliminates an entire class of risk. There is no server to breach, no transmission to intercept, no data retention policy to audit. The user's browser is the entire compute environment, and the user controls it completely.

This privacy property is not just a feature; it is often a compliance requirement. Regulations like GDPR, HIPAA, and various industry-specific rules impose strict controls on where sensitive data can be processed. In-browser analytics sidesteps these concerns entirely, because the data is never transferred to a third party.

Zero Server Costs

Running analytics in the browser eliminates server infrastructure costs. There is no compute to provision, no storage to pay for, no bandwidth to meter. For applications with unpredictable or bursty usage patterns, this is a significant economic advantage — the cost scales with users' own devices, not with a server bill.

For educational applications, this is transformative. An instructor can distribute a dataset and a query tool to hundreds of students without provisioning any infrastructure. Each student's browser does the work, and the instructor pays nothing.

Offline Analysis

Once the application is loaded, it works offline. Airplane mode, spotty Wi-Fi, or an air-gapped network — none of these prevent analysis. The data and the compute are both local, and the application's core functionality does not depend on network access.

This is valuable in field settings, in secure environments, and simply as a matter of reliability. An analyst working on a plane or in a coffee shop with unreliable Wi-Fi has the same experience as one in the office.

Educational SQL Playgrounds

For teaching SQL, the in-browser model is ideal. Students load their own data, experiment freely, and see results immediately. There is no signup, no server quota, no risk of accidentally affecting others' data. Mistakes are cheap and learning is fast.

The ability to share queries as URLs enables collaborative learning: an instructor can distribute a query, and students can reproduce it, modify it, and explore variations. The immediacy of the feedback loop — write query, see result, adjust — is exactly what effective learning requires.

Sharing and Reproducibility

Encoding queries in URLs makes analysis reproducible. A URL captures the exact query, and if the data source is accessible, anyone with the URL can reproduce the result. This is a lightweight alternative to notebook-based reproducibility, with the advantage of requiring no special software — just a browser.

Conclusion

DuckDB-Powered In-Browser Analytics represents a genuine shift in how data analysis can be delivered. By compiling a production-grade analytical database to WebAssembly and combining it with Apache Arrow for efficient data interchange, it achieves performance that approaches native applications while preserving the privacy and accessibility advantages of the browser.

The architecture is elegant in its simplicity: data stays local, computation happens where the data lives, and the server — when present — handles only metadata and coordination. This inversion of the traditional client-server model is enabled by decades of progress in browser capabilities: WebAssembly, Web Workers, IndexedDB, and the File API.

For sensitive datasets, offline analysis, educational contexts, and cost-sensitive applications, in-browser analytics offers a compelling alternative to server-based approaches. The technology is mature, the performance is real, and the privacy guarantees are absolute. As browsers continue to gain capabilities, the boundary between what can run locally and what requires a server will continue to shift — and DuckDB-WASM shows just how far that boundary has already moved.