Overview¶

arrowmodel converts Apache Arrow RecordBatch and Table objects directly into Pydantic v2 model instances – no intermediate Python dicts, no two-step materialisation. A tight Rust loop walks the Arrow buffers via the Arrow C Data Interface and hands you back typed models, roughly 2x faster than to_pylist() + Pydantic construction.

import pyarrow as pa
from arrowmodel import ArrowModel


class User(ArrowModel):
    id: int
    name: str
    score: float


batch = pa.record_batch(
    {
        "id": [1, 2, 3],
        "name": ["Alice", "Bob", "Carol"],
        "score": [9.5, 8.0, 7.3],
    }
)

users = User.convert(batch)
# [User(id=1, name='Alice', score=9.5), ...]

Dict-Free Conversion

Skips to_pylist() entirely. Arrow buffers go straight to model_construct calls in a single Rust loop.

~2x Faster

Roughly twice as fast as the pure-Python approach for flat schemas, with less memory allocation pressure.

Any Arrow Source

Accepts any Arrow-PyCapsule-compatible input – pyarrow, Polars, nanoarrow – via the Arrow C Data Interface.

Pydantic Aliases

Full alias resolution: validation_alias, alias, field name, and populate_by_name / validate_by_name.

Nested Models

Arrow Struct columns map automatically to nested Pydantic models, including Optional structs and deeply nested hierarchies.

Validated Mode

Choose between the fast path (model_construct, no validation) and the validated path (model_validate_json, full Pydantic checks).

Getting Started¶

Install with pip or uv:

pip

pip install arrowmodel

uv add arrowmodel

Pre-built wheels are provided for Linux (x86_64, aarch64), macOS on Apple Silicon (arm64) and Windows (x64), so no Rust toolchain is needed there. On any other platform, including Intel Macs, pip builds from source and needs Rust 1.83 or later (install it with rustup).

Then head to the Getting Started tutorial to convert your first batch.

Get Started