Guide

Getting Started With Python Excel AutomationDeep dive

Benchmark Python Excel Read and Write Speed

Published speed ratios are hypotheses about your file. Build a small harness that times openpyxl, calamine, Polars and xlsxwriter on the workbook you actually run.

Every recommendation about Python Excel performance comes with an unstated "on my file". Parse speed depends on how many cells are strings, whether the workbook carries a large shared-strings table, how much styling is present and whether formulas were cached — so the ratios published in blog posts, this one included, are hypotheses about your workbook rather than measurements of it. This guide, part of Choosing a Python Excel Library, builds a small harness that measures the libraries on the file you actually care about.

Prerequisites

Bash

pip install pandas polars openpyxl xlsxwriter python-calamine fastexcel

You also need a representative file. Not a synthetic sheet of random floats — a copy of the real export, because its shape is the thing being measured.

Generate a realistic test workbook

If you cannot use production data, build something with the same mix of types. A sheet that is all floats will flatter every parser equally and tell you nothing about the one you will run.

Python

import numpy as np
import pandas as pd

rows = 200_000
rng = np.random.default_rng(7)
frame = pd.DataFrame({
    "Order_ID": np.arange(1, rows + 1),
    "SKU": rng.choice([f"SKU-{n:04d}" for n in range(500)], rows),
    "Region": rng.choice(["North", "South", "East", "West"], rows),
    "Ordered": pd.date_range("2024-01-01", periods=rows, freq="min"),
    "Quantity": rng.integers(1, 40, rows),
    "Unit_Price": rng.normal(50, 12, rows).round(2),
})
frame.to_excel("bench.xlsx", sheet_name="Data", index=False)
print("wrote bench.xlsx")

Two string columns, a datetime column and three numeric ones is a reasonable approximation of a line-level export. Adjust the proportions towards whatever your real file looks like.

A harness that measures honestly

Three details separate a useful measurement from a misleading one: run each candidate more than once and keep the best time rather than the mean, warm the page cache before timing, and record the row count so a candidate that silently read half the sheet cannot look fast.

Python

import time
from statistics import median

def bench(name, fn, repeats=3):
    fn()                                    # warm the page cache, ignore this run
    times = []
    for _ in range(repeats):
        start = time.perf_counter()
        result = fn()
        times.append(time.perf_counter() - start)
    rows = getattr(result, "height", None) or len(result)
    print(f"{name:<26} {min(times):6.2f}s  best   {median(times):6.2f}s  median   {rows:,} rows")
    return min(times)

Keeping the minimum is deliberate. Anything that makes a run slower — another process, a garbage collection pause, the scheduler — is noise added on top of the true cost, so the fastest observed run is the closest estimate of it.

Measure the readers

Python

import pandas as pd
import polars as pl
from openpyxl import load_workbook

def read_openpyxl():
    return pd.read_excel("bench.xlsx", sheet_name="Data", engine="openpyxl")

def read_calamine():
    return pd.read_excel("bench.xlsx", sheet_name="Data", engine="calamine")

def read_polars():
    return pl.read_excel("bench.xlsx", sheet_name="Data")

def read_streaming():
    book = load_workbook("bench.xlsx", read_only=True, data_only=True)
    rows = list(book["Data"].iter_rows(min_row=2, values_only=True))
    book.close()
    return rows

for label, fn in [("pandas + openpyxl", read_openpyxl),
                  ("pandas + calamine", read_calamine),
                  ("polars.read_excel", read_polars),
                  ("openpyxl read_only", read_streaming)]:
    bench(label, fn)

The ordering is usually stable — the Rust parsers ahead of the Python one, streaming ahead of anything that builds a frame — but the size of the gap is not, and that is the number you are after. On a workbook with a very large shared-strings table the gap widens; on a small sheet of floats it can nearly vanish.

Measure the writers

Writing is the half people forget to measure, and the one where the differences are largest.

Python

import pandas as pd
import xlsxwriter

frame = pd.read_excel("bench.xlsx", sheet_name="Data", engine="calamine")

def write_openpyxl():
    frame.to_excel("out-openpyxl.xlsx", index=False, engine="openpyxl")
    return frame

def write_xlsxwriter():
    frame.to_excel("out-xlsxwriter.xlsx", index=False, engine="xlsxwriter")
    return frame

def write_constant_memory():
    book = xlsxwriter.Workbook("out-constant.xlsx", {"constant_memory": True})
    sheet = book.add_worksheet("Data")
    sheet.write_row(0, 0, list(frame.columns))
    for index, row in enumerate(frame.itertuples(index=False), start=1):
        sheet.write_row(index, 0, [str(v) if hasattr(v, "year") else v for v in row])
    book.close()
    return frame

for label, fn in [("to_excel + openpyxl", write_openpyxl),
                  ("to_excel + xlsxwriter", write_xlsxwriter),
                  ("xlsxwriter constant", write_constant_memory)]:
    bench(label, fn, repeats=2)

constant_memory mode changes the memory profile far more than the clock: it flushes each row as it is written, so the process footprint stops tracking the row count. The details are in Write a Million Rows to Excel with XlsxWriter in Constant Memory.

Measure memory, not just time

A job that finishes in four seconds and peaks at 3 GB will still be killed by a container limit. tracemalloc covers everything allocated through Python, which is exactly what openpyxl's cell objects are.

Python

import tracemalloc

def peak_mb(fn):
    tracemalloc.start()
    fn()
    _, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return peak / 1_048_576

print(f"openpyxl frame : {peak_mb(read_openpyxl):7.1f} MB")
print(f"read_only scan : {peak_mb(read_streaming):7.1f} MB")

For calamine and Polars, which allocate in Rust rather than through the Python allocator, tracemalloc under-reports badly. Use the resident set size instead:

Python

import resource

def peak_rss_mb():
    return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024   # KB on Linux

What to record alongside the numbers

A benchmark that is not reproducible next quarter is a screenshot, not a measurement. Four things belong in the same file as the timings: the library versions, the machine, the file's shape, and whether the run was warm or cold. Without them a result cannot be compared against the one you take after an upgrade, which is the comparison that eventually matters.

Python

import platform
import pandas as pd
import polars as pl
import openpyxl
import xlsxwriter

print({
    "python": platform.python_version(),
    "machine": platform.machine(),
    "pandas": pd.__version__,
    "polars": pl.__version__,
    "openpyxl": openpyxl.__version__,
    "xlsxwriter": xlsxwriter.__version__,
    "rows": len(frame),
    "columns": list(frame.columns),
})

Version drift in this ecosystem is real and mostly in your favour — calamine support arrived in pandas 2.2, and Polars' Excel reader has changed engines more than once — so a benchmark that records its versions turns an upgrade into a decision you can measure rather than a leap.

Reading the result, not just the ranking

The ranking is the least interesting part of the output. Three patterns in the numbers tell you what to do next.

If the streaming scan is close to the frame-building readers, the bottleneck is parsing rather than memory allocation, and the fix is a faster parser. If it is far ahead, the cost is in building the objects, and the fix is to avoid building them — stream, or prune columns.

If the writers are slower than the readers, the output is doing more work than the input, which usually means styling applied cell by cell. Setting a format on a column once, as Apply a Reusable Style Theme Across an Excel Report describes, collapses that cost.

And if every candidate is within a few percent of the others, the file is small enough that library choice is not your problem. Spend the effort on correctness instead — the checks in Validate an Excel Report Before Sending It will save more time than any parser will.

Common pitfalls

Symptom	Cause	Fix
Every library looks equally fast	The test file is too small for parse time to dominate	Benchmark at production scale, or generate ten times the rows
The first candidate is always slowest	Cold page cache on the first read	Warm with an untimed call, as the harness above does
A candidate is impossibly fast	It read fewer rows — wrong sheet, or a header offset	Print the row count with every result and compare them
Times swing by 50% between runs	Another process, or CPU frequency scaling	Take the minimum of several runs, and close other work
calamine is not faster	The file is small, or dominated by dates needing conversion	Check the column mix; the advantage is largest on wide string-heavy sheets
Memory numbers look wrong for Polars	Arrow buffers are allocated outside Python	Measure RSS rather than `tracemalloc`

Performance and scale

Two effects dominate anything else you will measure. Parsing cost scales with cells, not rows, so halving the columns with usecols halves the work as reliably as halving the rows — the technique in Read Specific Columns from Excel with Pandas. And repeated reads of an unchanged file are pure waste: converting once to Parquet turns a multi-second parse into a fraction of a second for every run afterwards.

That reframes the benchmark's real purpose. You are not looking for the fastest library so much as for the point where the format itself is the bottleneck — and past that point the answer is to stop reading .xlsx on every run, not to shave 20% off the parser.

Conclusion

Benchmark the file you have, not the one in someone else's article. A harness of fifteen lines — warm-up, repeats, minimum time, a printed row count — is enough to rank the readers and writers on your own data, and pairing it with a peak-memory number turns "which is faster" into the question that actually decides deployments: what fits in the container. Expect the Rust parsers ahead of openpyxl, streaming ahead of frame-building, and a converted columnar copy ahead of all of them.

Frequently asked questions

Why are my numbers different from every benchmark I read online? Because the file decides. A sheet of floats parses several times faster than the same number of cells holding formatted dates and strings, and a workbook with heavy styling carries a shared-strings table and a style index that both have to be parsed. Benchmark your own file; published ratios are only a starting hypothesis.

Should I use timeit instead of time.perf_counter? For an operation measured in seconds, perf_counter is fine and far easier to read. timeit earns its place for microsecond-scale calls where loop overhead would dominate — not for reading a workbook.

Does the first run being slower mean the benchmark is wrong? It means the operating system's page cache was cold. Run once to warm it, then measure — or measure both deliberately, because a scheduled job that reads a file straight off a network share never gets the warm case.

How do I measure memory as well as time? tracemalloc measures Python-allocated memory, which covers openpyxl's cell objects well. For libraries that allocate outside Python — calamine and Polars both do — read the process RSS from resource.getrusage or psutil instead.

Up one level: Choosing a Python Excel Library — the capability comparison these numbers support.
Speed Up pandas Excel Reads with the calamine Engine — the single biggest read-side change, in one keyword argument.
Speed Up openpyxl with read-only Mode — what streaming gives up in exchange for a flat memory curve.
Convert Excel Files to Parquet with Python — the change that beats every parser choice on repeated reads.
pandas vs Polars for Excel Workflows — the transform half of the same question.