Relyt ONE enables querying data stored on S3 object storage through external tables. The schema is automatically inferred from the data, simplifying table creation and greatly improving usability.
External tables support Parquet, JSON, and CSV formats. Powered by DuckDB’s read_parquet
, read_json
, and read_csv
functions, they deliver efficient data access. Built-in local and remote caching reduces latency and improves throughput for optimal query performance.
Syntax
CREATE TABLE <table_name> () USING duckdb WITH (
duckdb_external_location = 's3://xxxxxx',
duckdb_external_format = 'parquet', -- csv, json
duckdb_external_options = '' -- optional
);
Parameters
-
duckdb_external_location
: S3 path to your data. See Load Data for format details.
- Example:
s3://s3.us-west-1.amazonaws.com/bucket/path/dat.parquet accessid=xxxx secret=yyyy
-
duckdb_external_format
: Data format. Options: parquet
, csv
, json
-
duckdb_external_options
(optional): Additional parameters passed to DuckDB.
- Example: Include filename in results with
{"filename": true}
No need to define columns explicitly—Relyt ONE automatically samples the data to infer column names and types. After creation, use \d+ <table_name>
or SELECT * FROM <table> LIMIT 1
to inspect the schema.
External tables work like regular PostgreSQL tables. Queries involving external tables automatically leverage the DuckDB engine, with caching ensuring high performance regardless of S3 latency or bandwidth constraints.
Examples
Create a Parquet External Table
CREATE TABLE lineitem () USING duckdb WITH (
duckdb_external_location = 's3://s3.us-west-1.amazonaws.com/my_bucket/lineitem/*.parquet
accessid=xxxx secret=yyyy',
duckdb_external_format = 'parquet'
);
SELECT * FROM lineitem LIMIT 1;
Create a CSV External Table with Custom Column Names
CREATE TABLE nation () USING duckdb WITH (
duckdb_external_location = 's3://s3.us-west-1.amazonaws.com/my_bucket/csv/nation.tbl
accessid=xxxx secret=yyyy',
duckdb_external_format = 'csv',
duckdb_external_options = '{"column_names": ["n_nationkey","n_name","n_regionkey","n_comment"]}'
);
Create a JSON External Table
CREATE TABLE external_json () USING duckdb WITH (
duckdb_external_location = 's3://s3.us-west-1.amazonaws.com/my_bucket/json/a.tbl
accessid=xxxx secret=yyyy',
duckdb_external_format = 'json'
);
Verify the schema:
\d+ external_json
Table "public.external_json"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target
--------+------------------+-----------+----------+---------+----------+-------------+--------------
a | bigint | | | | plain | |
b | text | | | | extended | |
c | double precision | | | | plain | |
Access method: duckdb