Getting Started

Lakelet's philosophy is to be simple, fast, and be easy to use.

Install Lakelet

The quickest way is the installation script, which downloads the latest nightly binary into the current directory as ./lakelet, along with the example configuration config_demo.toml:

curl -fsSL https://lakelet.dev/install.sh | sh

The script supports Linux and macOS (x86_64 and aarch64).

Note that the x86_64 binaries target x86-64-v3 and require a CPU with AVX2 (Intel Haswell 2013+ / AMD Excavator+), and the Linux binaries require glibc 2.35 or newer.

Alternatively, download an archive directly from GitHub Releases — this is also the way to get the Windows build (lakelet-nightly-x86_64-pc-windows-msvc.zip).

Build from source

Lakelet requires a Rust toolchain. Build the release binary from the repository root:

git clone https://github.com/Smith-Cruise/Lakelet.git
cd Lakelet
cargo build --release
cp target/release/lakelet .

Create a Configuration File

Copy the example configuration and edit it for your environment:

cp config_demo.toml config.toml

A configuration file can contain the following top-level sections:

At least one external catalog is needed to query external tables. The built-in internal catalog is always available, even when no external catalog is configured.

The following is a minimal HMS configuration:

[[catalog.hms]]
name = "hms"
metastore-uri = "127.0.0.1:9083"

Add an s3-storage or oss-storage entry to the catalog when its tables use object storage:

[[catalog.hms]]
name = "hms"
metastore-uri = "127.0.0.1:9083"
s3-storage = { region = "us-east-1", endpoint = "http://127.0.0.1:9000", access-key = "admin", secret-key = "password", path-style-access = true }

See Server, Catalog, and Storage for the complete configuration reference.

Start Lakelet cli

Pass the configuration file with --config:

./lakelet --config config.toml

The configuration file is required for normal execution.

You can get more help by ./lakelet --help.

Arrow Flight SQL server

Lakelet can run as an Arrow Flight SQL server, including ADBC instead of the interactive REPL:

lakelet --config config.toml --flight-sql-server

The server listens on flight-sql-server-port under [server] (default 32010).

Note: Each flight SQL connection is a new fresh session, it will not share any SessionState. So USE state is discarded after every RPC and does not affect the next query even on the same ADBC connection.

Connect with ADBC (Python)

Lakelet works with the ADBC Flight SQL driver:

pip install adbc_driver_flightsql pyarrow

import adbc_driver_flightsql.dbapi as flight_sql

with flight_sql.connect("grpc://127.0.0.1:32010", autocommit=True) as conn:
    with conn.cursor() as cur:
        cur.execute("select 1 as a")
        print(cur.fetch_arrow_table())

Parameter binding (cur.execute(sql, params)) is not supported.

Every request may carry default-catalog and/or default-schema gRPC metadata headers.

import adbc_driver_flightsql.dbapi as flight_sql
from adbc_driver_flightsql import DatabaseOptions

HEADER = DatabaseOptions.RPC_CALL_HEADER_PREFIX.value

with flight_sql.connect(
    "grpc://127.0.0.1:32010",
    db_kwargs={
        HEADER + "default-catalog": "hive",
        HEADER + "default-schema": "sales",
    },
    autocommit=True,
) as conn:
    with conn.cursor() as cur:
        cur.execute("select * from orders limit 10")
        print(cur.fetch_arrow_table())

Connect with dft

Example using the datafusion-dft CLI/TUI (installed with the flightsql feature):

# ~/.config/dft/config.toml
[flightsql_client]
connection_url = "http://127.0.0.1:32010"

dft -c "select 1" --flightsql            # CLI
dft                                      # TUI: switch to the FlightSQL tab