Reading Excel with Polars and Arrow

pandas is not the only way to get a spreadsheet into Python any more. Polars — a DataFrame library built on Apache Arrow with a Rust core — reads Excel through the same fast calamine engine that pandas can now use, then runs transformations across every core in a query engine that plans the work before it executes. For reporting jobs that read a workbook and immediately aggregate, join or reshape it, the combination is markedly quicker and considerably lighter on memory. This topic covers reading and writing Excel from Polars, the Arrow-backed formats that pair with it, and — just as important — the cases where pandas or openpyxl remain the right answer. It extends Advanced Data Transformation and Cleaning with the newer half of the ecosystem.

Install and read your first workbook

Polars reads Excel through an optional extra, so install the reader alongside it:

pip install "polars[excel]"        # polars + fastexcel/calamine
pip install xlsxwriter             # only needed for write_excel

The read itself mirrors pandas closely enough that most code translates on sight:

import polars as pl

df = pl.read_excel("sales.xlsx")                       # first sheet
print(df.head())
print(df.schema)

shape: (5, 4)
┌────────────┬────────┬─────────┬──────────┐
│ order_date ┆ region ┆ product ┆ revenue  │
│ ---        ┆ ---    ┆ ---     ┆ ---      │
│ date       ┆ str    ┆ str     ┆ f64      │
└────────────┴────────┴─────────┴──────────┘

schema is worth printing on every new file. Polars infers types once, at read time, and being explicit about what it found saves the class of bug where a column of order numbers silently becomes a float. Selecting sheets works as it does in pandas:

q3 = pl.read_excel("sales.xlsx", sheet_name="Q3")
by_position = pl.read_excel("sales.xlsx", sheet_id=2)
everything = pl.read_excel("sales.xlsx", sheet_id=None)   # dict of DataFrames
print(list(everything))

The reader's own options are passed through — for example to skip a banner row or to force a column to text:

df = pl.read_excel(
    "sales.xlsx",
    read_options={"header_row": 2},
    schema_overrides={"order_id": pl.String},
)

schema_overrides is the Polars equivalent of pandas' dtype=, and it is the right tool for the perennial problem of leading zeros in reference codes — the same issue tackled from the pandas side in Convert Excel text columns to numbers with pandas.

Transform with expressions instead of loops

The reason to read into Polars is what comes next. Polars expressions describe a transformation, and the engine plans and parallelises it rather than executing statement by statement:

import polars as pl

df = pl.read_excel("sales.xlsx")

summary = (
    df.filter(pl.col("revenue") > 0)
      .with_columns(
          month=pl.col("order_date").dt.strftime("%Y-%m"),
          net=pl.col("revenue") * 0.8,
      )
      .group_by("region", "month")
      .agg(
          orders=pl.len(),
          revenue=pl.col("revenue").sum().round(2),
          best=pl.col("product").mode().first(),
      )
      .sort("region", "month")
)
print(summary)

Every step above is a column operation, so nothing iterates in Python. The equivalent pandas code is comparable in length but executes single-threaded per operation, and materialises an intermediate frame at each step. For a workbook of a few thousand rows the difference is imperceptible; over a few million rows joined against a database extract, it is the difference between a report that runs in a coffee break and one that does not.

Convert between Polars and pandas without paying for it

Adoption does not have to be all-or-nothing. Because both libraries speak Arrow, converting is close to free — no serialisation, and often no copy at all:

pdf = summary.to_pandas()          # hand off to existing formatting code
back = pl.from_pandas(pdf)         # and back again

That makes an incremental path realistic: read and reshape in Polars where the data is large, convert to pandas for the last mile, and keep every line of existing openpyxl or xlsxwriter styling code untouched. The formatting layer is covered across Formatting and Charting Excel Reports with Python.

Scan a folder lazily instead of reading it all

Polars' lazy API plans a whole pipeline before touching data, which lets it push filters down and read only the columns a query needs. Excel itself cannot be scanned lazily — the format has to be parsed in full — but the pattern still pays once workbooks are converted to Parquet:

from pathlib import Path

import polars as pl

# One-off: convert a folder of monthly workbooks to Parquet.
for path in sorted(Path("monthly").glob("*.xlsx")):
    pl.read_excel(path).write_parquet(path.with_suffix(".parquet"))

# Every run after that: scan lazily, filter early, collect once.
report = (
    pl.scan_parquet("monthly/*.parquet")
      .filter(pl.col("region") == "EMEA")
      .group_by("product")
      .agg(revenue=pl.col("revenue").sum())
      .sort("revenue", descending=True)
      .head(20)
      .collect()
)

The lazy scan reads only the region, product and revenue columns, and only the row groups that can contain EMEA rows. That is the single biggest speed-up available to a job that repeatedly re-reads the same historical workbooks — covered step by step in Convert Excel files to Parquet with Python.

Write Excel back out

Polars can write a workbook directly, driving xlsxwriter beneath the API, which is enough for most delivered reports:

summary.write_excel(
    "regional_summary.xlsx",
    worksheet="Summary",
    table_style="Table Style Medium 9",
    autofit=True,
    column_formats={"revenue": "#,##0.00", "net": "#,##0.00"},
    freeze_panes="A2",
)

That single call produces a formatted Excel table with number formats, autofitted columns and a frozen header — the same output that takes a couple of dozen lines through openpyxl. What it cannot do is edit an existing workbook or fill a template, because xlsxwriter only creates new files. For those, keep openpyxl: Populate an Excel template without losing formatting.

Know the limits before you commit

Polars is a data-processing library, not a spreadsheet library. It reads values, not presentation, and that shapes where it fits:

The honest summary is that Polars replaces the pandas half of a reporting job, not the openpyxl half. A typical modern pipeline reads with Polars, aggregates with Polars, and hands the result to xlsxwriter or openpyxl for presentation.

Clean the messy parts of a real sheet

Workbooks that come from people rather than systems need the same cleaning in Polars as in pandas — banner rows, trailing totals, text in numeric columns, inconsistent casing. The idioms differ, so here are the ones that come up in nearly every report:

import polars as pl

raw = pl.read_excel("messy.xlsx", read_options={"header_row": 3})

clean = (
    raw
    # drop rows that are entirely empty
    .filter(~pl.all_horizontal(pl.all().is_null()))
    # drop a trailing "Total" row that the exporter appended
    .filter(pl.col("region") != "Total")
    # normalise text columns in one pass
    .with_columns(
        pl.col(pl.String).str.strip_chars().str.to_titlecase(),
    )
    # coerce a column that arrived as "1,234.50" text
    .with_columns(
        pl.col("revenue").cast(pl.String)
          .str.replace_all(",", "")
          .cast(pl.Float64, strict=False)
          .alias("revenue"),
    )
    # give every column a snake_case name
    .rename(lambda name: name.strip().lower().replace(" ", "_"))
)
print(clean.null_count())

Two Polars-specific conveniences are worth noting. pl.col(pl.String) selects every column of a given type, so one expression normalises all text columns without naming them. And strict=False on a cast turns unparseable values into nulls rather than raising, which is the behaviour you want when a single stray footnote would otherwise abort the whole read. null_count() afterwards tells you how many values that cost — the Polars counterpart to the audit in Find and report missing values in an Excel file.

Join a workbook against a database extract

Reporting rarely uses one source. A common shape is a spreadsheet of manual adjustments joined onto a query result, and Polars joins are both fast and strict about what they do with unmatched rows:

import polars as pl

adjustments = pl.read_excel("adjustments.xlsx")
facts = pl.read_database_uri(
    query="SELECT order_id, region, revenue FROM orders WHERE order_date >= '2026-01-01'",
    uri="postgresql://reporting:secret@db.internal/analytics",
)

merged = facts.join(adjustments, on="order_id", how="left")
unmatched = adjustments.join(facts, on="order_id", how="anti")

print(f"{len(unmatched)} adjustment rows matched nothing")
final = merged.with_columns(
    revenue=pl.col("revenue") + pl.col("adjustment").fill_null(0.0)
)

The anti join is the piece worth stealing regardless of library: it names the rows from the spreadsheet that found no partner, which is exactly the reconciliation question a finance reviewer will ask. The pandas equivalent is in Find rows in one Excel file missing from another, and the database side in Export SQL query results to Excel with Python.

Measure it on your own data before switching

Published benchmarks are run on other people's files. The honest way to decide is a five-line measurement on the workbook your job actually reads, comparing the full path — read plus transformation — rather than the read alone:

"""Compare a real pipeline in both libraries."""
import time

import pandas as pd
import polars as pl

def timed(label, fn):
    start = time.perf_counter()
    result = fn()
    print(f"{label:24s} {time.perf_counter() - start:6.2f}s  rows={len(result)}")
    return result

timed("pandas openpyxl", lambda: (
    pd.read_excel("sales.xlsx", engine="openpyxl")
      .groupby("region", as_index=False)["revenue"].sum()
))
timed("pandas calamine", lambda: (
    pd.read_excel("sales.xlsx", engine="calamine")
      .groupby("region", as_index=False)["revenue"].sum()
))
timed("polars", lambda: (
    pl.read_excel("sales.xlsx").group_by("region").agg(pl.col("revenue").sum())
))

Run it three times and take the best, because the first read of a file is dominated by disk cache. If the parse dominates and the aggregation is trivial, switching engines inside pandas is the cheaper change; if the aggregation dominates, Polars is where the win is.

Read a folder of workbooks into one frame

Monthly exports arrive as one file per period, and the first job is nearly always to stack them. Polars concatenates with an explicit strategy for mismatched columns, which matters because a producer that adds a column in March should not silently drop the other months' data:

from pathlib import Path

import polars as pl

frames = []
for path in sorted(Path("monthly").glob("*.xlsx")):
    frame = pl.read_excel(path).with_columns(
        source_file=pl.lit(path.name),
        period=pl.lit(path.stem[-7:]),        # e.g. "2026-03"
    )
    frames.append(frame)

combined = pl.concat(frames, how="diagonal_relaxed")
print(combined.shape, combined.columns)

how="diagonal_relaxed" unions the columns, filling absent ones with nulls and widening types where they disagree — the behaviour you want for real exports. The stricter how="vertical" raises instead, which is the right choice when a schema change should stop the job. Tagging each row with its source file costs nothing and makes every later "where did this number come from?" question answerable; the pandas version of the same pattern is in Combine multiple Excel files into one with Python.

Fit it into a scheduled report job

Nothing about Polars changes the shape of an unattended job — it slots into the same ingest, transform, generate, deliver pipeline. What it does change is where the time goes, which affects how you structure the schedule:

"""A nightly job: convert, aggregate, write, hand off for delivery."""
from pathlib import Path

import polars as pl

RAW = Path("/srv/reports/incoming")
CACHE = Path("/srv/reports/cache")

def refresh_cache() -> None:
    """Parse each new workbook once; every later read is columnar."""
    CACHE.mkdir(exist_ok=True)
    for src in RAW.glob("*.xlsx"):
        dst = CACHE / f"{src.stem}.parquet"
        if not dst.exists() or dst.stat().st_mtime < src.stat().st_mtime:
            pl.read_excel(src).write_parquet(dst)

def build_summary() -> pl.DataFrame:
    return (
        pl.scan_parquet(CACHE / "*.parquet")
          .group_by("region", "product")
          .agg(revenue=pl.col("revenue").sum())
          .sort("revenue", descending=True)
          .collect()
    )

refresh_cache()
build_summary().write_excel("regional_summary.xlsx", autofit=True)

The mtime comparison means a rerun costs nothing for files that have not changed, so a job that is retried after a failure does not re-parse a whole archive. Delivery, logging and retries around this core are unchanged from any other report — see Automating Reporting Workflows and Retry a failed Excel report job in Python.

Watch the differences that catch pandas users out

Most of the translation is mechanical, but four behaviours differ enough to cause a wrong number rather than an error:

• No index. Polars has no row index at all. Anything you did with set_index, reset_index or index alignment becomes an explicit column and an explicit join.
• Nulls are not NaN. Polars distinguishes a missing value (null) from the float NaN. A column of floats can contain both, and is_null() does not match NaN — use is_nan() for that. This is the single most common source of surprise when porting a cleaning step.
• Strict types. A column is one type. Where pandas would happily hold a mixture in an object column, Polars forces a decision at read time — which is a feature, but it means schema_overrides earns its place in the read call.
• Expressions are lazy even in eager mode. pl.col("revenue") * 0.8 is a description, not a value; it only computes inside select, with_columns, filter or agg. Printing an expression shows the plan, not numbers.

import polars as pl

df = pl.DataFrame({"value": [1.0, None, float("nan")]})
print(df.select(
    nulls=pl.col("value").is_null().sum(),
    nans=pl.col("value").is_nan().sum(),
))

shape: (1, 2)
┌───────┬──────┐
│ nulls ┆ nans │
│ 1     ┆ 1    │
└───────┴──────┘

Knowing that distinction up front prevents the classic port bug: a fill_null(0) that leaves every NaN untouched, and a total that quietly comes out wrong. The equivalent decisions on the pandas side are covered in Handling Missing Data in Excel Reports.

What to install, and what each package is for

The dependency set is small, but each package has one job, and knowing which is which shortens the next debugging session:

Installing "polars[excel]" pulls the reader; "polars[all]" pulls everything above and is convenient in a container image where a missing extra means a failed nightly run rather than a quick local fix.

Key takeaways

• pl.read_excel() parses through the Rust calamine reader, which is consistently faster and leaner than pure-Python engines.
• Set schema_overrides at read time to control types instead of repairing them afterwards.
• Expressions are planned and executed across cores; the win grows with the size of the transformation, not just the size of the file.
• Polars and pandas convert through Arrow at negligible cost, so adoption can be incremental and reversible.
• Convert repeatedly-read workbooks to Parquet and scan them lazily — the largest speed-up available in most reporting jobs.
• write_excel() covers formatted output; templates, in-place edits and cell-level formatting still belong to openpyxl.

Frequently asked questions

Is Polars faster than pandas for Excel files? For the parsing itself, usually yes, because Polars reads through calamine — a Rust reader — rather than a pure-Python parser. The larger win is in what follows: group-bys, joins and filters over Arrow-backed columns run multi-threaded, so a heavy transformation after the read is where the difference shows.

Do I have to rewrite everything to adopt Polars? No. Polars and pandas convert to each other cheaply through Arrow, so a common pattern is to read and transform in Polars, then hand a pandas DataFrame to whatever formatting code you already have.

Can Polars write a styled Excel report?write_excel drives xlsxwriter under the hood, so it can apply number formats, autofit columns, add an Excel table and even a conditional format. For fine-grained control over an existing workbook you still want openpyxl.

Does Polars handle multiple sheets? Yes. Pass sheet_name or sheet_id, or None to get a dictionary of DataFrames keyed by sheet name, in the same shape pandas returns.

When should I convert Excel data to Parquet? As soon as the same workbook is read more than once. Parquet keeps types, compresses well, and loads an order of magnitude faster, so a nightly conversion turns a slow Excel read into a fast columnar read for every downstream job.

Conclusion

Polars gives Excel automation a faster front end without asking you to abandon the tools that make workbooks presentable. Read through calamine, control types at the boundary, express transformations as column operations, and convert anything you will read twice to Parquet. Then hand the result to xlsxwriter or openpyxl for the formatting, charts and templates that only a spreadsheet library can produce.

Related

• Up: Advanced Data Transformation and Cleaning — the section this topic extends, and where the pandas equivalents live.
• Read an Excel file with polars.read_excel — the reader in full, including sheets, headers and schema control.
• Speed up pandas Excel reads with the calamine engine — the same fast parser without leaving pandas.
• Write a Polars DataFrame to Excel with formatting — write_excel options for a delivered report.
• Convert Excel files to Parquet with Python — the conversion that makes every later read cheap.
• Sibling topics: Working with Large Excel Files in Python and Cleaning Excel Data with pandas — the pandas-side techniques these tools complement.
• pandas vs Polars for Excel Workflows — where the two genuinely differ, and how to move between them.