[{"data":1,"prerenderedAt":2176},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars":2145,"ancestors:\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars":2154,"children:\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars":2159,"related:\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars":2160},{"id":4,"title":5,"body":6,"dateModified":2110,"datePublished":2110,"description":2111,"extension":2112,"faq":2113,"keyword":2122,"kind":2123,"meta":2124,"navigation":139,"path":2133,"seo":2134,"slug":2137,"stem":2138,"topics":2139,"type":2143,"__hash__":2144},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars\u002Findex.md","Join and Pivot Excel Data with Polars",{"type":7,"value":8,"toc":2094},"minimark",[9,40,45,73,107,111,240,267,271,416,437,580,584,587,677,687,691,786,808,812,942,970,1273,1277,1284,1451,1477,1481,1488,1548,1559,1563,1756,1765,1891,1895,1915,1919,1996,2000,2029,2033,2039,2045,2051,2057,2061,2090],[10,11,12,13,17,18,21,22,25,26,29,30,33,34,39],"p",{},"Polars is a fast DataFrame library with a strict, expression-based API, and it reads Excel files through the Rust-based ",[14,15,16],"code",{},"fastexcel"," engine. Once the data is loaded, the two jobs that dominate report scripts — joining tables and reshaping them into a pivot — look a little different from pandas: there is no index, joins name their keys explicitly and can validate them, and ",[14,19,20],{},"pivot"," takes ",[14,23,24],{},"on",", ",[14,27,28],{},"index"," and ",[14,31,32],{},"values"," arguments. This page builds a small sales report from a workbook with three sheets: it joins the order lines to products and regions, checks for codes that do not match, aggregates, pivots quarters into columns, and writes the result back to Excel with formatting. It belongs to ",[35,36,38],"a",{"href":37},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002F","Reading Excel with Polars and Arrow",".",[41,42,44],"h2",{"id":43},"prerequisites","Prerequisites",[46,47,52],"pre",{"className":48,"code":49,"language":50,"meta":51,"style":51},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install \"polars[excel]\" xlsxwriter\n","bash","",[14,53,54],{"__ignoreMap":51},[55,56,59,63,67,70],"span",{"class":57,"line":58},"line",1,[55,60,62],{"class":61},"sMTad","pip",[55,64,66],{"class":65},"srMev"," install",[55,68,69],{"class":65}," \"polars[excel]\"",[55,71,72],{"class":65}," xlsxwriter\n",[10,74,75,76,79,80,82,83,86,87,90,91,94,95,98,99,102,103,106],{},"The ",[14,77,78],{},"excel"," extra installs ",[14,81,16],{}," for reading; XlsxWriter is used by ",[14,84,85],{},"write_excel",". The examples use Polars 1.x. ",[14,88,89],{},"sales_book.xlsx"," has an ",[14,92,93],{},"Orders"," sheet (order lines with store and product codes), a ",[14,96,97],{},"Products"," sheet with prices and a ",[14,100,101],{},"Stores"," sheet with regions. One order references store ",[14,104,105],{},"S9",", which is not in the store list.",[41,108,110],{"id":109},"step-1-read-the-sheets","Step 1 — Read the sheets",[46,112,116],{"className":113,"code":114,"language":115,"meta":51,"style":51},"language-python shiki shiki-themes github-light github-dark-high-contrast","import polars as pl\n\norders = pl.read_excel(\"sales_book.xlsx\", sheet_name=\"Orders\")\nproducts = pl.read_excel(\"sales_book.xlsx\", sheet_name=\"Products\")\nstores = pl.read_excel(\"sales_book.xlsx\", sheet_name=\"Stores\")\nprint(orders.schema)\nprint(orders.head(3))\n","python",[14,117,118,134,141,170,193,216,226],{"__ignoreMap":51},[55,119,120,124,128,131],{"class":57,"line":58},[55,121,123],{"class":122},"s-kum","import",[55,125,127],{"class":126},"skGVy"," polars ",[55,129,130],{"class":122},"as",[55,132,133],{"class":126}," pl\n",[55,135,137],{"class":57,"line":136},2,[55,138,140],{"emptyLinePlaceholder":139},true,"\n",[55,142,144,147,150,153,156,158,162,164,167],{"class":57,"line":143},3,[55,145,146],{"class":126},"orders ",[55,148,149],{"class":122},"=",[55,151,152],{"class":126}," pl.read_excel(",[55,154,155],{"class":65},"\"sales_book.xlsx\"",[55,157,25],{"class":126},[55,159,161],{"class":160},"sa561","sheet_name",[55,163,149],{"class":122},[55,165,166],{"class":65},"\"Orders\"",[55,168,169],{"class":126},")\n",[55,171,173,176,178,180,182,184,186,188,191],{"class":57,"line":172},4,[55,174,175],{"class":126},"products ",[55,177,149],{"class":122},[55,179,152],{"class":126},[55,181,155],{"class":65},[55,183,25],{"class":126},[55,185,161],{"class":160},[55,187,149],{"class":122},[55,189,190],{"class":65},"\"Products\"",[55,192,169],{"class":126},[55,194,196,199,201,203,205,207,209,211,214],{"class":57,"line":195},5,[55,197,198],{"class":126},"stores ",[55,200,149],{"class":122},[55,202,152],{"class":126},[55,204,155],{"class":65},[55,206,25],{"class":126},[55,208,161],{"class":160},[55,210,149],{"class":122},[55,212,213],{"class":65},"\"Stores\"",[55,215,169],{"class":126},[55,217,219,223],{"class":57,"line":218},6,[55,220,222],{"class":221},"sP0c6","print",[55,224,225],{"class":126},"(orders.schema)\n",[55,227,229,231,234,237],{"class":57,"line":228},7,[55,230,222],{"class":221},[55,232,233],{"class":126},"(orders.head(",[55,235,236],{"class":221},"3",[55,238,239],{"class":126},"))\n",[10,241,242,243,246,247,250,251,254,255,258,259,262,263,39],{},"Polars infers a schema per column — ",[14,244,245],{},"OrderDate"," arrives as a ",[14,248,249],{},"date"," (the cells hold midnight times), ",[14,252,253],{},"Qty"," as an integer, codes as strings — and, unlike pandas, does not invent an index. Reading every sheet at once with ",[14,256,257],{},"sheet_id=0"," returns a dictionary of frames, the counterpart of pandas' ",[14,260,261],{},"sheet_name=None","; reading is covered in ",[35,264,266],{"href":265},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fread-an-excel-file-with-polars-read-excel\u002F","Read an Excel File with Polars read_excel",[41,268,270],{"id":269},"step-2-join-with-key-validation","Step 2 — Join with key validation",[46,272,274],{"className":113,"code":273,"language":115,"meta":51,"style":51},"lines = (orders\n         .join(products, on=\"SKU\", how=\"left\", validate=\"m:1\")\n         .join(stores, on=\"StoreID\", how=\"left\", validate=\"m:1\")\n         .with_columns(Revenue=pl.col(\"Qty\") * pl.col(\"Price\")))\nprint(lines.select(\"OrderDate\", \"StoreID\", \"Region\", \"Product\", \"Qty\", \"Revenue\"))\n",[14,275,276,286,320,350,381],{"__ignoreMap":51},[55,277,278,281,283],{"class":57,"line":58},[55,279,280],{"class":126},"lines ",[55,282,149],{"class":122},[55,284,285],{"class":126}," (orders\n",[55,287,288,291,293,295,298,300,303,305,308,310,313,315,318],{"class":57,"line":136},[55,289,290],{"class":126},"         .join(products, ",[55,292,24],{"class":160},[55,294,149],{"class":122},[55,296,297],{"class":65},"\"SKU\"",[55,299,25],{"class":126},[55,301,302],{"class":160},"how",[55,304,149],{"class":122},[55,306,307],{"class":65},"\"left\"",[55,309,25],{"class":126},[55,311,312],{"class":160},"validate",[55,314,149],{"class":122},[55,316,317],{"class":65},"\"m:1\"",[55,319,169],{"class":126},[55,321,322,325,327,329,332,334,336,338,340,342,344,346,348],{"class":57,"line":143},[55,323,324],{"class":126},"         .join(stores, ",[55,326,24],{"class":160},[55,328,149],{"class":122},[55,330,331],{"class":65},"\"StoreID\"",[55,333,25],{"class":126},[55,335,302],{"class":160},[55,337,149],{"class":122},[55,339,307],{"class":65},[55,341,25],{"class":126},[55,343,312],{"class":160},[55,345,149],{"class":122},[55,347,317],{"class":65},[55,349,169],{"class":126},[55,351,352,355,358,360,363,366,369,372,375,378],{"class":57,"line":172},[55,353,354],{"class":126},"         .with_columns(",[55,356,357],{"class":160},"Revenue",[55,359,149],{"class":122},[55,361,362],{"class":126},"pl.col(",[55,364,365],{"class":65},"\"Qty\"",[55,367,368],{"class":126},") ",[55,370,371],{"class":122},"*",[55,373,374],{"class":126}," pl.col(",[55,376,377],{"class":65},"\"Price\"",[55,379,380],{"class":126},")))\n",[55,382,383,385,388,391,393,395,397,400,402,405,407,409,411,414],{"class":57,"line":195},[55,384,222],{"class":221},[55,386,387],{"class":126},"(lines.select(",[55,389,390],{"class":65},"\"OrderDate\"",[55,392,25],{"class":126},[55,394,331],{"class":65},[55,396,25],{"class":126},[55,398,399],{"class":65},"\"Region\"",[55,401,25],{"class":126},[55,403,404],{"class":65},"\"Product\"",[55,406,25],{"class":126},[55,408,365],{"class":65},[55,410,25],{"class":126},[55,412,413],{"class":65},"\"Revenue\"",[55,415,239],{"class":126},[10,417,418,421,422,425,426,428,429,432,433,436],{},[14,419,420],{},"how=\"left\""," keeps every order line, and ",[14,423,424],{},"validate=\"m:1\""," asserts that each SKU and each store ID appears only once in the lookup tables; if a duplicate product row crept into the ",[14,427,97],{}," sheet, Polars would raise an error instead of silently duplicating order lines. The S9 order comes through with an empty ",[14,430,431],{},"Region",". ",[14,434,435],{},"with_columns"," adds the revenue as an expression — Polars' way of saying \"compute this column from those\", evaluated efficiently across the whole frame.",[438,439,447,448,447,452,447,456,447,464,447,474,447,485,447,491,447,496,447,502,447,507,447,511,447,518,447,523,447,526,447,530,447,533,447,536,447,541,447,546,447,550,447,554,447,558,447,561,447,566,447,573,447,577],"svg",{"viewBox":440,"role":441,"ariaLabelledBy":442,"xmlns":445,"style":446},"4 0 752 142","img",[443,444],"pl-joins-t","pl-joins-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[449,450,451],"title",{"id":443},"Building enriched order lines with two validated joins",[453,454,455],"desc",{"id":444},"The Orders sheet is joined to Products on SKU and to Stores on StoreID, both as left joins validated many-to-one, and a Revenue column is added as quantity times price.",[457,458],"rect",{"x":459,"y":460,"width":461,"height":462,"fill":463},"4","0","752","142","#ffffff",[457,465],{"x":466,"y":467,"width":468,"height":469,"rx":470,"fill":471,"stroke":472,"style":473},"20.0","16.0","153.0","110.5","12","#e7ebef","var(--line,#cdd5e6)","stroke-width:2px",[475,476,478],"g",{"transform":477},"translate(84.5,37.8) scale(1.000)",[479,480],"path",{"d":481,"fill":482,"stroke":483,"style":484},"M3 5h18v14H3z M3 10h18 M3 15h18 M9 5v14","none","var(--text,#172033)","stroke-width:1.80px;stroke-linecap:round;stroke-linejoin:round",[486,487,93],"text",{"x":488,"y":489,"style":490},"96.5","84.2","font-size:13.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle",[486,492,495],{"x":488,"y":493,"style":494},"103.8","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","one row per line",[57,497],{"x1":498,"y1":499,"x2":500,"y2":499,"stroke":501,"style":473},"178.0","71.2","200.0","var(--brand,#5b5cf0)",[503,504],"polygon",{"points":505,"fill":506},"204.0,71.2 195.0,76.2 195.0,66.2","#5b5cf0",[457,508],{"x":509,"y":467,"width":468,"height":469,"rx":470,"fill":510,"stroke":501,"style":473},"209.0","#f0f4ff",[475,512,514],{"transform":513},"translate(273.5,37.8) scale(1.000)",[479,515],{"d":516,"fill":482,"stroke":517,"style":484},"M10 14a4 4 0 0 0 5.7 0l3-3a4 4 0 0 0-5.7-5.7l-1 1 M14 10a4 4 0 0 0-5.7 0l-3 3a4 4 0 0 0 5.7 5.7l1-1","var(--brand-strong,#4338ca)",[486,519,522],{"x":520,"y":489,"style":521},"285.5","font-size:13.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","+ Products",[486,524,525],{"x":520,"y":493,"style":494},"on SKU, validate m:1",[57,527],{"x1":528,"y1":499,"x2":529,"y2":499,"stroke":501,"style":473},"367.0","389.0",[503,531],{"points":532,"fill":506},"393.0,71.2 384.0,76.2 384.0,66.2",[457,534],{"x":535,"y":467,"width":468,"height":469,"rx":470,"fill":510,"stroke":501,"style":473},"398.0",[475,537,539],{"transform":538},"translate(462.5,29.0) scale(1.000)",[479,540],{"d":516,"fill":482,"stroke":517,"style":484},[486,542,545],{"x":543,"y":544,"style":521},"474.5","75.5","+ Stores",[486,547,549],{"x":543,"y":548,"style":494},"95.0","on StoreID, validate",[486,551,553],{"x":543,"y":552,"style":494},"112.5","m:1",[57,555],{"x1":556,"y1":499,"x2":557,"y2":499,"stroke":501,"style":473},"556.0","578.0",[503,559],{"points":560,"fill":506},"582.0,71.2 573.0,76.2 573.0,66.2",[457,562],{"x":563,"y":467,"width":468,"height":469,"rx":470,"fill":564,"stroke":565,"style":473},"587.0","#d9f4f1","var(--teal,#0f9488)",[475,567,569],{"transform":568},"translate(651.5,37.8) scale(1.000)",[479,570],{"d":571,"fill":482,"stroke":572,"style":484},"M5 9h14 M4 15h14 M10 4l-2 16 M16 4l-2 16","var(--teal-ink,#0b6157)",[486,574,357],{"x":575,"y":489,"style":576},"663.5","font-size:13.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle",[486,578,579],{"x":575,"y":493,"style":494},"Qty × Price",[41,581,583],{"id":582},"step-3-find-unmatched-keys-with-an-anti-join","Step 3 — Find unmatched keys with an anti join",[10,585,586],{},"A left join hides unmatched rows as nulls; an anti join lists them directly:",[46,588,590],{"className":113,"code":589,"language":115,"meta":51,"style":51},"unknown_stores = orders.join(stores, on=\"StoreID\", how=\"anti\")\nunknown_products = orders.join(products, on=\"SKU\", how=\"anti\")\nprint(\"orders with unknown stores:\\n\", unknown_stores)\nprint(\"orders with unknown products:\", unknown_products.height)\n",[14,591,592,619,645,665],{"__ignoreMap":51},[55,593,594,597,599,602,604,606,608,610,612,614,617],{"class":57,"line":58},[55,595,596],{"class":126},"unknown_stores ",[55,598,149],{"class":122},[55,600,601],{"class":126}," orders.join(stores, ",[55,603,24],{"class":160},[55,605,149],{"class":122},[55,607,331],{"class":65},[55,609,25],{"class":126},[55,611,302],{"class":160},[55,613,149],{"class":122},[55,615,616],{"class":65},"\"anti\"",[55,618,169],{"class":126},[55,620,621,624,626,629,631,633,635,637,639,641,643],{"class":57,"line":136},[55,622,623],{"class":126},"unknown_products ",[55,625,149],{"class":122},[55,627,628],{"class":126}," orders.join(products, ",[55,630,24],{"class":160},[55,632,149],{"class":122},[55,634,297],{"class":65},[55,636,25],{"class":126},[55,638,302],{"class":160},[55,640,149],{"class":122},[55,642,616],{"class":65},[55,644,169],{"class":126},[55,646,647,649,652,655,659,662],{"class":57,"line":143},[55,648,222],{"class":221},[55,650,651],{"class":126},"(",[55,653,654],{"class":65},"\"orders with unknown stores:",[55,656,658],{"class":657},"sSjpA","\\n",[55,660,661],{"class":65},"\"",[55,663,664],{"class":126},", unknown_stores)\n",[55,666,667,669,671,674],{"class":57,"line":172},[55,668,222],{"class":221},[55,670,651],{"class":126},[55,672,673],{"class":65},"\"orders with unknown products:\"",[55,675,676],{"class":126},", unknown_products.height)\n",[10,678,679,682,683,39],{},[14,680,681],{},"how=\"anti\""," returns the rows of the left frame whose key has no match on the right — here the single S9 order. Checking anti joins before reporting is cheap and catches master-data gaps that would otherwise show up as a mysterious \"null\" region in a pivot. The pandas technique is shown in ",[35,684,686],{"href":685},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Ffind-rows-in-one-excel-file-missing-from-another\u002F","Find Rows in One Excel File Missing from Another",[41,688,690],{"id":689},"step-4-group-and-aggregate","Step 4 — Group and aggregate",[46,692,694],{"className":113,"code":693,"language":115,"meta":51,"style":51},"by_region = (lines\n             .filter(pl.col(\"Region\").is_not_null())\n             .group_by(\"Region\")\n             .agg(Orders=pl.len(), Units=pl.col(\"Qty\").sum(), Revenue=pl.col(\"Revenue\").sum())\n             .sort(\"Revenue\", descending=True))\nprint(by_region)\n",[14,695,696,706,716,725,760,779],{"__ignoreMap":51},[55,697,698,701,703],{"class":57,"line":58},[55,699,700],{"class":126},"by_region ",[55,702,149],{"class":122},[55,704,705],{"class":126}," (lines\n",[55,707,708,711,713],{"class":57,"line":136},[55,709,710],{"class":126},"             .filter(pl.col(",[55,712,399],{"class":65},[55,714,715],{"class":126},").is_not_null())\n",[55,717,718,721,723],{"class":57,"line":143},[55,719,720],{"class":126},"             .group_by(",[55,722,399],{"class":65},[55,724,169],{"class":126},[55,726,727,730,732,734,737,740,742,744,746,749,751,753,755,757],{"class":57,"line":172},[55,728,729],{"class":126},"             .agg(",[55,731,93],{"class":160},[55,733,149],{"class":122},[55,735,736],{"class":126},"pl.len(), ",[55,738,739],{"class":160},"Units",[55,741,149],{"class":122},[55,743,362],{"class":126},[55,745,365],{"class":65},[55,747,748],{"class":126},").sum(), ",[55,750,357],{"class":160},[55,752,149],{"class":122},[55,754,362],{"class":126},[55,756,413],{"class":65},[55,758,759],{"class":126},").sum())\n",[55,761,762,765,767,769,772,774,777],{"class":57,"line":195},[55,763,764],{"class":126},"             .sort(",[55,766,413],{"class":65},[55,768,25],{"class":126},[55,770,771],{"class":160},"descending",[55,773,149],{"class":122},[55,775,776],{"class":221},"True",[55,778,239],{"class":126},[55,780,781,783],{"class":57,"line":218},[55,782,222],{"class":221},[55,784,785],{"class":126},"(by_region)\n",[10,787,788,791,792,795,796,799,800,803,804,807],{},[14,789,790],{},"group_by"," takes the key, ",[14,793,794],{},"agg"," takes named expressions, and nothing happens implicitly: unmatched rows are filtered out explicitly rather than forming a null group. Polars does not sort groups by default — and does not even keep them in first-seen order unless you pass ",[14,797,798],{},"maintain_order=True"," — so ",[14,801,802],{},"sort"," states the order the report wants. The North region, with six order lines worth 892.50, comes first; South follows with three lines and 454.00. ",[14,805,806],{},"pl.len()"," counts rows per group and returns an unsigned integer, which writes to Excel like any other number.",[41,809,811],{"id":810},"step-5-pivot-quarters-into-columns","Step 5 — Pivot quarters into columns",[46,813,815],{"className":113,"code":814,"language":115,"meta":51,"style":51},"quarterly = (lines\n             .filter(pl.col(\"Region\").is_not_null())\n             .with_columns(Quarter=pl.lit(\"Q\") + pl.col(\"OrderDate\").dt.quarter().cast(pl.String))\n             .pivot(on=\"Quarter\", index=\"Region\", values=\"Revenue\", aggregate_function=\"sum\")\n             .fill_null(0)\n             .select(\"Region\", \"Q1\", \"Q2\", \"Q3\"))\nprint(quarterly)\n",[14,816,817,826,834,862,902,911,935],{"__ignoreMap":51},[55,818,819,822,824],{"class":57,"line":58},[55,820,821],{"class":126},"quarterly ",[55,823,149],{"class":122},[55,825,705],{"class":126},[55,827,828,830,832],{"class":57,"line":136},[55,829,710],{"class":126},[55,831,399],{"class":65},[55,833,715],{"class":126},[55,835,836,839,842,844,847,850,852,855,857,859],{"class":57,"line":143},[55,837,838],{"class":126},"             .with_columns(",[55,840,841],{"class":160},"Quarter",[55,843,149],{"class":122},[55,845,846],{"class":126},"pl.lit(",[55,848,849],{"class":65},"\"Q\"",[55,851,368],{"class":126},[55,853,854],{"class":122},"+",[55,856,374],{"class":126},[55,858,390],{"class":65},[55,860,861],{"class":126},").dt.quarter().cast(pl.String))\n",[55,863,864,867,869,871,874,876,878,880,882,884,886,888,890,892,895,897,900],{"class":57,"line":172},[55,865,866],{"class":126},"             .pivot(",[55,868,24],{"class":160},[55,870,149],{"class":122},[55,872,873],{"class":65},"\"Quarter\"",[55,875,25],{"class":126},[55,877,28],{"class":160},[55,879,149],{"class":122},[55,881,399],{"class":65},[55,883,25],{"class":126},[55,885,32],{"class":160},[55,887,149],{"class":122},[55,889,413],{"class":65},[55,891,25],{"class":126},[55,893,894],{"class":160},"aggregate_function",[55,896,149],{"class":122},[55,898,899],{"class":65},"\"sum\"",[55,901,169],{"class":126},[55,903,904,907,909],{"class":57,"line":195},[55,905,906],{"class":126},"             .fill_null(",[55,908,460],{"class":221},[55,910,169],{"class":126},[55,912,913,916,918,920,923,925,928,930,933],{"class":57,"line":218},[55,914,915],{"class":126},"             .select(",[55,917,399],{"class":65},[55,919,25],{"class":126},[55,921,922],{"class":65},"\"Q1\"",[55,924,25],{"class":126},[55,926,927],{"class":65},"\"Q2\"",[55,929,25],{"class":126},[55,931,932],{"class":65},"\"Q3\"",[55,934,239],{"class":126},[55,936,937,939],{"class":57,"line":228},[55,938,222],{"class":221},[55,940,941],{"class":126},"(quarterly)\n",[10,943,944,946,947,949,950,952,953,955,956,958,959,962,963,966,967,39],{},[14,945,20],{}," turns the distinct values of ",[14,948,24],{}," into columns, one row per ",[14,951,28],{}," value, aggregating ",[14,954,32],{}," with ",[14,957,894],{},". Column order follows the order in which values first appear, so the final ",[14,960,961],{},"select"," fixes Q1–Q3 in sequence, and ",[14,964,965],{},"fill_null(0)"," turns region\u002Fquarter combinations without sales into zeros. pandas users will recognise this as ",[14,968,969],{},"pivot_table(index=\"Region\", columns=\"Quarter\", values=\"Revenue\", aggfunc=\"sum\", fill_value=0)",[438,971,447,977,447,980,447,983,447,988,447,995,447,999,447,1005,447,1009,447,1013,447,1017,447,1021,447,1025,447,1030,447,1033,447,1036,447,1038,447,1040,447,1042,447,1044,447,1047,447,1051,447,1053,447,1057,447,1059,447,1062,447,1064,447,1067,447,1070,447,1073,447,1075,447,1077,447,1079,447,1082,447,1084,447,1087,447,1090,447,1093,447,1095,447,1098,447,1100,447,1102,447,1104,447,1107,447,1110,447,1114,447,1116,447,1118,447,1120,447,1123,447,1125,447,1128,447,1131,447,1135,447,1137,447,1139,447,1141,447,1143,447,1145,447,1148,447,1151,447,1154,447,1157,447,1161,447,1164,447,1168,447,1171,447,1175,447,1179,447,1181,447,1184,447,1186,447,1188,447,1190,447,1192,447,1194,447,1196,447,1198,447,1200,447,1202,447,1204,447,1206,447,1208,447,1210,447,1212,447,1214,447,1216,447,1218,447,1220,447,1222,447,1224,447,1226,447,1228,447,1230,447,1232,447,1234,447,1237,447,1239,447,1241,447,1247,447,1251,447,1256,447,1259,447,1267],{"viewBox":972,"role":441,"ariaLabelledBy":973,"xmlns":445,"style":976},"10 0 740 248",[974,975],"pl-pivot-t","pl-pivot-d","width:100%;max-width:740px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[449,978,979],{"id":974},"Revenue by region pivoted into quarter columns",[453,981,982],{"id":975},"On the left the long table of region, quarter and revenue. On the right one row per region, North and South, with columns Q1, Q2 and Q3 holding summed revenue and zero where a region had no sales.",[457,984],{"x":985,"y":460,"width":986,"height":987,"fill":463},"10","740","248",[457,989],{"x":990,"y":991,"width":992,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"26.1","40.0","32.0","24.0","stroke-width:1px",[457,996],{"x":997,"y":991,"width":998,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"58.1","61.4",[486,1000,1004],{"x":1001,"y":1002,"style":1003},"88.8","57.0","font-size:11px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","A",[457,1006],{"x":1007,"y":991,"width":1008,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"119.5","65.4",[486,1010,1012],{"x":1011,"y":1002,"style":1003},"152.2","B",[457,1014],{"x":1015,"y":991,"width":1016,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"185.0","71.3",[486,1018,1020],{"x":1019,"y":1002,"style":1003},"220.6","C",[457,1022],{"x":990,"y":1023,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},"64.0","28.0",[486,1026,1029],{"x":1027,"y":1028,"style":1003},"42.1","83.0","1",[457,1031],{"x":997,"y":1023,"width":998,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},"stroke-width:1.5px",[486,1034,431],{"x":1001,"y":1028,"style":1035},"font-size:10.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle",[457,1037],{"x":1007,"y":1023,"width":1008,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},[486,1039,841],{"x":1011,"y":1028,"style":1035},[457,1041],{"x":1015,"y":1023,"width":1016,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},[486,1043,357],{"x":1019,"y":1028,"style":1035},[457,1045],{"x":990,"y":1046,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},"92.0",[486,1048,1050],{"x":1027,"y":1049,"style":1003},"111.0","2",[457,1052],{"x":997,"y":1046,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1054,1056],{"x":1001,"y":1049,"style":1055},"font-size:10.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","North",[457,1058],{"x":1007,"y":1046,"width":1008,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1060,1061],{"x":1011,"y":1049,"style":1055},"Q1",[457,1063],{"x":1015,"y":1046,"width":1016,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1065,1066],{"x":1019,"y":1049,"style":1055},"356",[457,1068],{"x":990,"y":1069,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},"120.0",[486,1071,236],{"x":1027,"y":1072,"style":1003},"139.0",[457,1074],{"x":997,"y":1069,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1076,1056],{"x":1001,"y":1072,"style":1055},[457,1078],{"x":1007,"y":1069,"width":1008,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1080,1081],{"x":1011,"y":1072,"style":1055},"Q2",[457,1083],{"x":1015,"y":1069,"width":1016,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1085,1086],{"x":1019,"y":1072,"style":1055},"288",[457,1088],{"x":990,"y":1089,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},"148.0",[486,1091,459],{"x":1027,"y":1092,"style":1003},"167.0",[457,1094],{"x":997,"y":1089,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1096,1097],{"x":1001,"y":1092,"style":1055},"South",[457,1099],{"x":1007,"y":1089,"width":1008,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1101,1061],{"x":1011,"y":1092,"style":1055},[457,1103],{"x":1015,"y":1089,"width":1016,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1105,1106],{"x":1019,"y":1092,"style":1055},"140",[457,1108],{"x":990,"y":1109,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},"176.0",[486,1111,1113],{"x":1027,"y":1112,"style":1003},"195.0","5",[457,1115],{"x":997,"y":1109,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1117,1056],{"x":1001,"y":1112,"style":1055},[457,1119],{"x":1007,"y":1109,"width":1008,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1121,1122],{"x":1011,"y":1112,"style":1055},"Q3",[457,1124],{"x":1015,"y":1109,"width":1016,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1126,1127],{"x":1019,"y":1112,"style":1055},"248.5",[457,1129],{"x":990,"y":1130,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},"204.0",[486,1132,1134],{"x":1027,"y":1133,"style":1003},"223.0","6",[457,1136],{"x":997,"y":1130,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1138,1097],{"x":1001,"y":1133,"style":1055},[457,1140],{"x":1007,"y":1130,"width":1008,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1142,1122],{"x":1011,"y":1133,"style":1055},[457,1144],{"x":1015,"y":1130,"width":1016,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1146,1147],{"x":1019,"y":1133,"style":1055},"182",[457,1149],{"x":1150,"y":991,"width":992,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"493.9",[457,1152],{"x":1153,"y":991,"width":998,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"525.9",[486,1155,1004],{"x":1156,"y":1002,"style":1003},"556.6",[457,1158],{"x":1159,"y":991,"width":1160,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"587.3","45.1",[486,1162,1012],{"x":1163,"y":1002,"style":1003},"609.9",[457,1165],{"x":1166,"y":991,"width":1167,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"632.4","45.3",[486,1169,1020],{"x":1170,"y":1002,"style":1003},"655.1",[457,1172],{"x":1173,"y":991,"width":1174,"height":993,"rx":460,"fill":471,"stroke":472,"style":994},"677.7","56.1",[486,1176,1178],{"x":1177,"y":1002,"style":1003},"705.8","D",[457,1180],{"x":1150,"y":1023,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},[486,1182,1029],{"x":1183,"y":1028,"style":1003},"509.9",[457,1185],{"x":1153,"y":1023,"width":998,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},[486,1187,431],{"x":1156,"y":1028,"style":1035},[457,1189],{"x":1159,"y":1023,"width":1160,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},[486,1191,1061],{"x":1163,"y":1028,"style":1035},[457,1193],{"x":1166,"y":1023,"width":1167,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},[486,1195,1081],{"x":1170,"y":1028,"style":1035},[457,1197],{"x":1173,"y":1023,"width":1174,"height":1024,"rx":460,"fill":510,"stroke":501,"style":1032},[486,1199,1122],{"x":1177,"y":1028,"style":1035},[457,1201],{"x":1150,"y":1046,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},[486,1203,1050],{"x":1183,"y":1049,"style":1003},[457,1205],{"x":1153,"y":1046,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1207,1056],{"x":1156,"y":1049,"style":1055},[457,1209],{"x":1159,"y":1046,"width":1160,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1211,1066],{"x":1163,"y":1049,"style":1055},[457,1213],{"x":1166,"y":1046,"width":1167,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1215,1086],{"x":1170,"y":1049,"style":1055},[457,1217],{"x":1173,"y":1046,"width":1174,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1219,1127],{"x":1177,"y":1049,"style":1055},[457,1221],{"x":1150,"y":1069,"width":992,"height":1024,"rx":460,"fill":471,"stroke":472,"style":994},[486,1223,236],{"x":1183,"y":1072,"style":1003},[457,1225],{"x":1153,"y":1069,"width":998,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1227,1097],{"x":1156,"y":1072,"style":1055},[457,1229],{"x":1159,"y":1069,"width":1160,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1231,1106],{"x":1163,"y":1072,"style":1055},[457,1233],{"x":1166,"y":1069,"width":1167,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1235,1236],{"x":1170,"y":1072,"style":1055},"132",[457,1238],{"x":1173,"y":1069,"width":1174,"height":1024,"rx":460,"fill":463,"stroke":472,"style":994},[486,1240,1147],{"x":1177,"y":1072,"style":1055},[486,1242,1246],{"x":1243,"y":1244,"style":1245},"141.2","29.0","font-size:12.5px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","long",[486,1248,1250],{"x":1249,"y":1244,"style":1245},"613.9","wide",[57,1252],{"x1":1253,"y1":1254,"x2":1255,"y2":1254,"stroke":501,"style":473},"266.3","94.0","479.9",[503,1257],{"points":1258,"fill":506},"483.9,94.0 474.9,99.0 474.9,89.0",[457,1260],{"x":1261,"y":1262,"width":1263,"height":1264,"rx":1265,"fill":463,"stroke":472,"style":1266},"296.0","61.0","158.1","22.0","11","stroke-width:1.2px",[486,1268,1272],{"x":1269,"y":1270,"style":1271},"375.1","76.0","font-size:11.5px;font-weight:400;fill:var(--brand-strong,#4338ca);text-anchor:middle;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace","pivot(on=\"Quarter\")",[41,1274,1276],{"id":1275},"step-6-the-same-pipeline-lazily","Step 6 — The same pipeline, lazily",[10,1278,1279,1280,1283],{},"For larger inputs, describe the whole pipeline first and let Polars optimise it. ",[14,1281,1282],{},"read_excel"," itself is eager, but the frames can be turned into lazy frames:",[46,1285,1287],{"className":113,"code":1286,"language":115,"meta":51,"style":51},"plan = (orders.lazy()\n        .join(products.lazy(), on=\"SKU\", how=\"left\")\n        .join(stores.lazy(), on=\"StoreID\", how=\"left\")\n        .filter(pl.col(\"Region\").is_not_null())\n        .group_by(\"Region\", \"Product\")\n        .agg(Revenue=(pl.col(\"Qty\") * pl.col(\"Price\")).sum())\n        .sort(\"Region\", \"Revenue\", descending=[False, True]))\nprint(plan.explain()[:300])\nresult = plan.collect()\nprint(result)\n",[14,1288,1289,1299,1320,1341,1350,1363,1388,1418,1432,1443],{"__ignoreMap":51},[55,1290,1291,1294,1296],{"class":57,"line":58},[55,1292,1293],{"class":126},"plan ",[55,1295,149],{"class":122},[55,1297,1298],{"class":126}," (orders.lazy()\n",[55,1300,1301,1304,1306,1308,1310,1312,1314,1316,1318],{"class":57,"line":136},[55,1302,1303],{"class":126},"        .join(products.lazy(), ",[55,1305,24],{"class":160},[55,1307,149],{"class":122},[55,1309,297],{"class":65},[55,1311,25],{"class":126},[55,1313,302],{"class":160},[55,1315,149],{"class":122},[55,1317,307],{"class":65},[55,1319,169],{"class":126},[55,1321,1322,1325,1327,1329,1331,1333,1335,1337,1339],{"class":57,"line":143},[55,1323,1324],{"class":126},"        .join(stores.lazy(), ",[55,1326,24],{"class":160},[55,1328,149],{"class":122},[55,1330,331],{"class":65},[55,1332,25],{"class":126},[55,1334,302],{"class":160},[55,1336,149],{"class":122},[55,1338,307],{"class":65},[55,1340,169],{"class":126},[55,1342,1343,1346,1348],{"class":57,"line":172},[55,1344,1345],{"class":126},"        .filter(pl.col(",[55,1347,399],{"class":65},[55,1349,715],{"class":126},[55,1351,1352,1355,1357,1359,1361],{"class":57,"line":195},[55,1353,1354],{"class":126},"        .group_by(",[55,1356,399],{"class":65},[55,1358,25],{"class":126},[55,1360,404],{"class":65},[55,1362,169],{"class":126},[55,1364,1365,1368,1370,1372,1375,1377,1379,1381,1383,1385],{"class":57,"line":218},[55,1366,1367],{"class":126},"        .agg(",[55,1369,357],{"class":160},[55,1371,149],{"class":122},[55,1373,1374],{"class":126},"(pl.col(",[55,1376,365],{"class":65},[55,1378,368],{"class":126},[55,1380,371],{"class":122},[55,1382,374],{"class":126},[55,1384,377],{"class":65},[55,1386,1387],{"class":126},")).sum())\n",[55,1389,1390,1393,1395,1397,1399,1401,1403,1405,1408,1411,1413,1415],{"class":57,"line":228},[55,1391,1392],{"class":126},"        .sort(",[55,1394,399],{"class":65},[55,1396,25],{"class":126},[55,1398,413],{"class":65},[55,1400,25],{"class":126},[55,1402,771],{"class":160},[55,1404,149],{"class":122},[55,1406,1407],{"class":126},"[",[55,1409,1410],{"class":221},"False",[55,1412,25],{"class":126},[55,1414,776],{"class":221},[55,1416,1417],{"class":126},"]))\n",[55,1419,1421,1423,1426,1429],{"class":57,"line":1420},8,[55,1422,222],{"class":221},[55,1424,1425],{"class":126},"(plan.explain()[:",[55,1427,1428],{"class":221},"300",[55,1430,1431],{"class":126},"])\n",[55,1433,1435,1438,1440],{"class":57,"line":1434},9,[55,1436,1437],{"class":126},"result ",[55,1439,149],{"class":122},[55,1441,1442],{"class":126}," plan.collect()\n",[55,1444,1446,1448],{"class":57,"line":1445},10,[55,1447,222],{"class":221},[55,1449,1450],{"class":126},"(result)\n",[10,1452,1453,1456,1457,1459,1460,1464,1465,1468,1469,1472,1473,39],{},[14,1454,1455],{},"explain()"," shows the optimised plan, and it is instructive here: Polars noticed that a left join followed by a filter on ",[14,1458,431],{}," being not null is the same as an ",[1461,1462,1463],"em",{},"inner"," join, rewrote it as one, and carries only the four columns the aggregation needs through the joins. ",[14,1466,1467],{},"collect()"," runs it. For a few thousand rows the gain is negligible; for millions it is substantial, and the code stays the same. If the source data is large and refreshed often, converting it to Parquet and starting with ",[14,1470,1471],{},"pl.scan_parquet"," makes the whole pipeline lazy, as in ",[35,1474,1476],{"href":1475},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fconvert-excel-files-to-parquet-with-python\u002F","Convert Excel Files to Parquet with Python",[41,1478,1480],{"id":1479},"step-7-unpivot-back-to-long-form","Step 7 — Unpivot back to long form",[10,1482,1483,1484,1487],{},"Wide tables suit reading; long tables suit charts and further joins. ",[14,1485,1486],{},"unpivot"," reverses the pivot:",[46,1489,1491],{"className":113,"code":1490,"language":115,"meta":51,"style":51},"long = quarterly.unpivot(index=\"Region\", variable_name=\"Quarter\", value_name=\"Revenue\").sort(\"Region\", \"Quarter\")\nprint(long)\n",[14,1492,1493,1538],{"__ignoreMap":51},[55,1494,1495,1497,1500,1503,1505,1507,1509,1511,1514,1516,1518,1520,1523,1525,1527,1530,1532,1534,1536],{"class":57,"line":58},[55,1496,1246],{"class":160},[55,1498,1499],{"class":122}," =",[55,1501,1502],{"class":126}," quarterly.unpivot(",[55,1504,28],{"class":160},[55,1506,149],{"class":122},[55,1508,399],{"class":65},[55,1510,25],{"class":126},[55,1512,1513],{"class":160},"variable_name",[55,1515,149],{"class":122},[55,1517,873],{"class":65},[55,1519,25],{"class":126},[55,1521,1522],{"class":160},"value_name",[55,1524,149],{"class":122},[55,1526,413],{"class":65},[55,1528,1529],{"class":126},").sort(",[55,1531,399],{"class":65},[55,1533,25],{"class":126},[55,1535,873],{"class":65},[55,1537,169],{"class":126},[55,1539,1540,1542,1544,1546],{"class":57,"line":136},[55,1541,222],{"class":221},[55,1543,651],{"class":126},[55,1545,1246],{"class":160},[55,1547,169],{"class":126},[10,1549,1550,1551,1554,1555,39],{},"Each region\u002Fquarter pair becomes a row again, ready for a chart that plots one line per region, or for joining to a targets table keyed on region and quarter. In pandas the same step is ",[14,1552,1553],{},"melt(id_vars=\"Region\", var_name=\"Quarter\", value_name=\"Revenue\")",", covered in ",[35,1556,1558],{"href":1557},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcreating-pivot-tables-from-excel-data\u002Funpivot-a-wide-excel-sheet-with-pandas-melt\u002F","Unpivot a Wide Excel Sheet with pandas melt",[41,1560,1562],{"id":1561},"step-8-write-a-formatted-workbook","Step 8 — Write a formatted workbook",[46,1564,1566],{"className":113,"code":1565,"language":115,"meta":51,"style":51},"import xlsxwriter\n\nwith xlsxwriter.Workbook(\"regional_report.xlsx\") as wb:\n    by_region.write_excel(wb, worksheet=\"By region\", autofit=True,\n                          column_formats={\"Revenue\": \"#,##0.00\"}, table_style=\"Table Style Light 9\")\n    quarterly.write_excel(wb, worksheet=\"By quarter\", autofit=True,\n                          column_formats={\"Q1\": \"#,##0\", \"Q2\": \"#,##0\", \"Q3\": \"#,##0\"})\n    unknown_stores.write_excel(wb, worksheet=\"Unknown stores\", autofit=True)\nprint(\"sheets:\", pl.read_excel(\"regional_report.xlsx\", sheet_id=0).keys())\n",[14,1567,1568,1574,1578,1596,1621,1652,1674,1708,1730],{"__ignoreMap":51},[55,1569,1570,1572],{"class":57,"line":58},[55,1571,123],{"class":122},[55,1573,72],{"class":126},[55,1575,1576],{"class":57,"line":136},[55,1577,140],{"emptyLinePlaceholder":139},[55,1579,1580,1583,1586,1589,1591,1593],{"class":57,"line":143},[55,1581,1582],{"class":122},"with",[55,1584,1585],{"class":126}," xlsxwriter.Workbook(",[55,1587,1588],{"class":65},"\"regional_report.xlsx\"",[55,1590,368],{"class":126},[55,1592,130],{"class":122},[55,1594,1595],{"class":126}," wb:\n",[55,1597,1598,1601,1604,1606,1609,1611,1614,1616,1618],{"class":57,"line":172},[55,1599,1600],{"class":126},"    by_region.write_excel(wb, ",[55,1602,1603],{"class":160},"worksheet",[55,1605,149],{"class":122},[55,1607,1608],{"class":65},"\"By region\"",[55,1610,25],{"class":126},[55,1612,1613],{"class":160},"autofit",[55,1615,149],{"class":122},[55,1617,776],{"class":221},[55,1619,1620],{"class":126},",\n",[55,1622,1623,1626,1628,1631,1633,1636,1639,1642,1645,1647,1650],{"class":57,"line":195},[55,1624,1625],{"class":160},"                          column_formats",[55,1627,149],{"class":122},[55,1629,1630],{"class":126},"{",[55,1632,413],{"class":65},[55,1634,1635],{"class":126},": ",[55,1637,1638],{"class":65},"\"#,##0.00\"",[55,1640,1641],{"class":126},"}, ",[55,1643,1644],{"class":160},"table_style",[55,1646,149],{"class":122},[55,1648,1649],{"class":65},"\"Table Style Light 9\"",[55,1651,169],{"class":126},[55,1653,1654,1657,1659,1661,1664,1666,1668,1670,1672],{"class":57,"line":218},[55,1655,1656],{"class":126},"    quarterly.write_excel(wb, ",[55,1658,1603],{"class":160},[55,1660,149],{"class":122},[55,1662,1663],{"class":65},"\"By quarter\"",[55,1665,25],{"class":126},[55,1667,1613],{"class":160},[55,1669,149],{"class":122},[55,1671,776],{"class":221},[55,1673,1620],{"class":126},[55,1675,1676,1678,1680,1682,1684,1686,1689,1691,1693,1695,1697,1699,1701,1703,1705],{"class":57,"line":228},[55,1677,1625],{"class":160},[55,1679,149],{"class":122},[55,1681,1630],{"class":126},[55,1683,922],{"class":65},[55,1685,1635],{"class":126},[55,1687,1688],{"class":65},"\"#,##0\"",[55,1690,25],{"class":126},[55,1692,927],{"class":65},[55,1694,1635],{"class":126},[55,1696,1688],{"class":65},[55,1698,25],{"class":126},[55,1700,932],{"class":65},[55,1702,1635],{"class":126},[55,1704,1688],{"class":65},[55,1706,1707],{"class":126},"})\n",[55,1709,1710,1713,1715,1717,1720,1722,1724,1726,1728],{"class":57,"line":1420},[55,1711,1712],{"class":126},"    unknown_stores.write_excel(wb, ",[55,1714,1603],{"class":160},[55,1716,149],{"class":122},[55,1718,1719],{"class":65},"\"Unknown stores\"",[55,1721,25],{"class":126},[55,1723,1613],{"class":160},[55,1725,149],{"class":122},[55,1727,776],{"class":221},[55,1729,169],{"class":126},[55,1731,1732,1734,1736,1739,1742,1744,1746,1749,1751,1753],{"class":57,"line":1434},[55,1733,222],{"class":221},[55,1735,651],{"class":126},[55,1737,1738],{"class":65},"\"sheets:\"",[55,1740,1741],{"class":126},", pl.read_excel(",[55,1743,1588],{"class":65},[55,1745,25],{"class":126},[55,1747,1748],{"class":160},"sheet_id",[55,1750,149],{"class":122},[55,1752,460],{"class":221},[55,1754,1755],{"class":126},").keys())\n",[10,1757,1758,1760,1761,39],{},[14,1759,85],{}," writes each frame as an Excel table with a style, number formats per column and fitted widths; passing an open XlsxWriter workbook puts several frames into one file. Including the unknown-store sheet keeps the data-quality issue visible to the people who can fix it. More options are in ",[35,1762,1764],{"href":1763},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fwrite-a-polars-dataframe-to-excel-with-formatting\u002F","Write a Polars DataFrame to Excel with Formatting",[438,1766,447,1771,447,1774,447,1777,447,1780,447,1783,447,1787,447,1793,447,1796,447,1800,447,1804,447,1809,447,1811,447,1815,447,1817,447,1820,447,1823,447,1827,447,1829,447,1832,447,1834,447,1837,447,1840,447,1844,447,1846,447,1850,447,1852,447,1855,447,1858,447,1862,447,1864,447,1868,447,1870,447,1873,447,1876,447,1880,447,1882,447,1886,447,1888],{"viewBox":1767,"role":441,"ariaLabelledBy":1768,"xmlns":445,"style":446},"4 0 752 242",[1769,1770],"pl-vs-pandas-t","pl-vs-pandas-d",[449,1772,1773],{"id":1769},"Polars operations and their pandas counterparts",[453,1775,1776],{"id":1770},"Polars join with how and validate corresponds to pandas merge with how and validate. Polars anti join corresponds to a merge with indicator and a left_only filter. group_by with agg corresponds to groupby with named aggregation. pivot with on, index and values corresponds to pivot_table with columns, index and values. unpivot corresponds to melt.",[457,1778],{"x":459,"y":460,"width":461,"height":1779,"fill":463},"242",[457,1781],{"x":466,"y":467,"width":1782,"height":991,"rx":460,"fill":463,"stroke":472,"style":994},"137.2",[457,1784],{"x":1785,"y":467,"width":1786,"height":991,"rx":460,"fill":510,"stroke":501,"style":1266},"157.2","291.4",[486,1788,1792],{"x":1789,"y":1790,"style":1791},"302.9","41.0","font-size:13px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Polars",[457,1794],{"x":1795,"y":467,"width":1786,"height":991,"rx":460,"fill":510,"stroke":501,"style":1266},"448.6",[486,1797,1799],{"x":1798,"y":1790,"style":1791},"594.3","pandas",[457,1801],{"x":466,"y":1802,"width":1782,"height":1803,"rx":460,"fill":471,"stroke":472,"style":994},"56.0","34.0",[486,1805,1808],{"x":992,"y":1806,"style":1807},"78.0","font-size:12.5px;font-weight:700;fill:var(--text,#172033);text-anchor:start","Join with checks",[457,1810],{"x":1785,"y":1802,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1812,1814],{"x":1789,"y":1813,"style":494},"77.0","join(how, validate=\"m:1\")",[457,1816],{"x":1795,"y":1802,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1818,1819],{"x":1798,"y":1813,"style":494},"merge(how, validate=\"many_to_one\")",[457,1821],{"x":466,"y":1822,"width":1782,"height":1803,"rx":460,"fill":471,"stroke":472,"style":994},"90.0",[486,1824,1826],{"x":992,"y":1825,"style":1807},"112.0","Unmatched rows",[457,1828],{"x":1785,"y":1822,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1830,1831],{"x":1789,"y":1049,"style":494},"join(how=\"anti\")",[457,1833],{"x":1795,"y":1822,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1835,1836],{"x":1798,"y":1049,"style":494},"merge(indicator=True) + left_only",[457,1838],{"x":466,"y":1839,"width":1782,"height":1803,"rx":460,"fill":471,"stroke":472,"style":994},"124.0",[486,1841,1843],{"x":992,"y":1842,"style":1807},"146.0","Aggregate",[457,1845],{"x":1785,"y":1839,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1847,1849],{"x":1789,"y":1848,"style":494},"145.0","group_by().agg(name=expr)",[457,1851],{"x":1795,"y":1839,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1853,1854],{"x":1798,"y":1848,"style":494},"groupby().agg(name=(col, func))",[457,1856],{"x":466,"y":1857,"width":1782,"height":1803,"rx":460,"fill":471,"stroke":472,"style":994},"158.0",[486,1859,1861],{"x":992,"y":1860,"style":1807},"180.0","Wide pivot",[457,1863],{"x":1785,"y":1857,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1865,1867],{"x":1789,"y":1866,"style":494},"179.0","pivot(on, index, values)",[457,1869],{"x":1795,"y":1857,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1871,1872],{"x":1798,"y":1866,"style":494},"pivot_table(columns, index, values)",[457,1874],{"x":466,"y":1875,"width":1782,"height":1803,"rx":460,"fill":471,"stroke":472,"style":994},"192.0",[486,1877,1879],{"x":992,"y":1878,"style":1807},"214.0","Back to long",[457,1881],{"x":1785,"y":1875,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1883,1885],{"x":1789,"y":1884,"style":494},"213.0","unpivot(index=…)",[457,1887],{"x":1795,"y":1875,"width":1786,"height":1803,"rx":460,"fill":463,"stroke":472,"style":994},[486,1889,1890],{"x":1798,"y":1884,"style":494},"melt(id_vars=…)",[41,1892,1894],{"id":1893},"performance-notes","Performance notes",[10,1896,1897,1898,1900,1901,1904,1905,1907,1908,29,1911,1914],{},"Reading is usually the slowest step, and ",[14,1899,16],{}," is among the fastest Excel readers available to Python; like pandas' ",[14,1902,1903],{},"calamine"," engine, it is built on the Rust ",[14,1906,1903],{}," crate. After that, Polars' joins, group-bys and pivots run in parallel across CPU cores, which shows on tables of hundreds of thousands of rows and more. For workbooks of a few thousand rows like this one, any tool is instant, and the reasons to choose Polars are its explicitness — no index, validated joins, plans you can inspect — rather than speed. If the rest of a project uses pandas, ",[14,1909,1910],{},"df.to_pandas()",[14,1912,1913],{},"pl.from_pandas(df)"," move data across cheaply.",[41,1916,1918],{"id":1917},"common-pitfalls","Common pitfalls",[1920,1921,1922,1930,1939,1948,1957,1973,1990],"ul",{},[1923,1924,1925,1929],"li",{},[1926,1927,1928],"strong",{},"Expecting an index."," Polars has none; join and group on named columns.",[1923,1931,1932,1935,1936,1938],{},[1926,1933,1934],{},"Silent row duplication."," Use ",[14,1937,424],{}," on lookups so duplicate keys raise.",[1923,1940,1941,1944,1945,1947],{},[1926,1942,1943],{},"Null groups."," Unmatched rows form a null group in ",[14,1946,790],{},"; filter or label them explicitly.",[1923,1949,1950,1953,1954,1956],{},[1926,1951,1952],{},"Pivot column order."," Columns appear in first-seen order; ",[14,1955,961],{}," them in the order you need.",[1923,1958,1959,1962,1963,1965,1966,1969,1970,39],{},[1926,1960,1961],{},"Older Polars examples."," ",[14,1964,20],{}," used to take ",[14,1967,1968],{},"columns=","; in current Polars the argument is ",[14,1971,1972],{},"on=",[1923,1974,1975,1978,1979,1982,1983,1986,1987,1989],{},[1926,1976,1977],{},"Pivoting a lazy frame."," A lazy plan must know its output columns in advance, so ",[14,1980,1981],{},"LazyFrame.pivot"," (in recent Polars) requires ",[14,1984,1985],{},"on_columns=[\"Q1\", \"Q2\", \"Q3\"]","; on a DataFrame, ",[14,1988,20],{}," discovers them from the data. Listing them explicitly also fixes their order.",[1923,1991,1992,1995],{},[1926,1993,1994],{},"Integer and float arithmetic."," Multiplying an integer quantity by a float price gives a float; cast explicitly when the output must stay integer.",[41,1997,1999],{"id":1998},"conclusion","Conclusion",[10,2001,2002,2003,2006,2007,29,2009,2011,2012,2014,2015,2018,2019,2022,2023,2025,2026,2028],{},"Read each sheet with ",[14,2004,2005],{},"pl.read_excel",", join lookups with ",[14,2008,420],{},[14,2010,424],{},", list unmatched keys with ",[14,2013,681],{},", aggregate with ",[14,2016,2017],{},"group_by().agg()",", and reshape with ",[14,2020,2021],{},"pivot(on=..., index=..., values=..., aggregate_function=...)",", fixing the column order with ",[14,2024,961],{},". Switch to lazy frames when the data grows, and write the results with ",[14,2027,85],{},", including any data-quality sheets so problems stay visible.",[41,2030,2032],{"id":2031},"frequently-asked-questions","Frequently asked questions",[10,2034,2035,2038],{},[1926,2036,2037],{},"How do I join two Excel sheets with Polars?","\nRead each sheet with pl.read_excel(path, sheet_name=...), then call left.join(right, on=\"key\", how=\"left\"). Add validate=\"m:1\" to make sure each key appears only once in the lookup table.",[10,2040,2041,2044],{},[1926,2042,2043],{},"How do I create a pivot table in Polars?","\nUse df.pivot(on=\"Quarter\", index=\"Region\", values=\"Revenue\", aggregate_function=\"sum\"), then fill_null(0) for combinations without data and select the columns in the order you want.",[10,2046,2047,2050],{},[1926,2048,2049],{},"How do I find rows that do not match in a Polars join?","\nUse an anti join: left.join(right, on=\"key\", how=\"anti\") returns the rows of left whose key is not present in right.",[10,2052,2053,2056],{},[1926,2054,2055],{},"Can Polars write Excel files with formatting?","\nYes. DataFrame.write_excel writes each frame as an Excel table and accepts number formats per column, a table style, autofit and conditional formats. It uses XlsxWriter under the hood.",[41,2058,2060],{"id":2059},"related","Related",[1920,2062,2063,2069,2076,2083],{},[1923,2064,2065,2066,2068],{},"Up one level: ",[35,2067,38],{"href":37}," — fast Excel reads and Arrow-based tools.",[1923,2070,2071,2075],{},[35,2072,2074],{"href":2073},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fclean-and-aggregate-excel-data-with-polars\u002F","Clean and Aggregate Excel Data with Polars"," — cleaning steps before the join.",[1923,2077,2078,2082],{},[35,2079,2081],{"href":2080},"\u002Fgetting-started-with-python-excel-automation\u002Fchoosing-a-python-excel-library\u002Fpandas-vs-polars-for-excel-workflows\u002F","pandas vs Polars for Excel Workflows"," — when to choose which.",[1923,2084,2085,2089],{},[35,2086,2088],{"href":2087},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcreating-pivot-tables-from-excel-data\u002Fcreate-pivot-table-from-excel-with-pandas\u002F","Create a Pivot Table from Excel with pandas"," — the pandas version.",[2091,2092,2093],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":51,"searchDepth":136,"depth":136,"links":2095},[2096,2097,2098,2099,2100,2101,2102,2103,2104,2105,2106,2107,2108,2109],{"id":43,"depth":136,"text":44},{"id":109,"depth":136,"text":110},{"id":269,"depth":136,"text":270},{"id":582,"depth":136,"text":583},{"id":689,"depth":136,"text":690},{"id":810,"depth":136,"text":811},{"id":1275,"depth":136,"text":1276},{"id":1479,"depth":136,"text":1480},{"id":1561,"depth":136,"text":1562},{"id":1893,"depth":136,"text":1894},{"id":1917,"depth":136,"text":1918},{"id":1998,"depth":136,"text":1999},{"id":2031,"depth":136,"text":2032},{"id":2059,"depth":136,"text":2060},"2026-10-02","Combine and reshape workbook data with Polars: read several sheets, join them with validate to catch duplicate keys, find unmatched rows with an anti join, group and aggregate, pivot quarters into columns, unpivot back, run the pipeline lazily, and write a formatted result with write_excel — with the pandas equivalents alongside.","md",[2114,2116,2118,2120],{"q":2037,"a":2115},"Read each sheet with pl.read_excel(path, sheet_name=...), then call left.join(right, on=\"key\", how=\"left\"). Add validate=\"m:1\" to make sure each key appears only once in the lookup table.",{"q":2043,"a":2117},"Use df.pivot(on=\"Quarter\", index=\"Region\", values=\"Revenue\", aggregate_function=\"sum\"), then fill_null(0) for combinations without data and select the columns in the order you want.",{"q":2049,"a":2119},"Use an anti join: left.join(right, on=\"key\", how=\"anti\") returns the rows of left whose key is not present in right.",{"q":2055,"a":2121},"Yes. DataFrame.write_excel writes each frame as an Excel table and accepts number formats per column, a table style, autofit and conditional formats. It uses XlsxWriter under the hood.","join and pivot excel data with polars",null,{"breadcrumb":2125},[2126,2129,2132],{"name":2127,"item":2128},"Home","\u002F",{"name":2130,"item":2131},"Advanced Data Transformation and Cleaning for Excel Automation","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":38,"item":37},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars",{"title":2135,"description":2136},"Join and Pivot Excel Data with Polars (read_excel, pivot)","Polars for Excel data: read_excel sheets, join with how and validate=\"m:1\", anti joins for unmatched keys, group_by aggregations, pivot(on, index, values, aggregate_function), unpivot, lazy pipelines and write_excel output.","join-and-pivot-excel-data-with-polars","advanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fjoin-and-pivot-excel-data-with-polars\u002Findex",[2140,2141,2142],"polars","joins-and-lookups","pivots-and-summaries","how-to","jEYluYyAAJB0G8fZxmmzrT_Y5V7Gwc_ehp8CNdE2y3M",[2146,2150],{"title":2147,"path":2148,"stem":2149,"children":-1},"Fix Schema and Type Errors When Reading Excel with Polars","\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Ffix-schema-and-type-errors-when-reading-excel-with-polars","advanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Ffix-schema-and-type-errors-when-reading-excel-with-polars\u002Findex",{"title":2151,"path":2152,"stem":2153,"children":-1},"Read an Excel File with polars.read_excel","\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fread-an-excel-file-with-polars-read-excel","advanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fread-an-excel-file-with-polars-read-excel\u002Findex",[2155,2157],{"path":2156,"title":2130},"\u002Fadvanced-data-transformation-and-cleaning",{"path":2158,"title":38},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow",[],[2161,2164,2167,2170,2172,2173],{"path":2162,"title":2163,"kind":2143},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fclean-and-aggregate-excel-data-with-polars","Clean and Aggregate Excel Data with Polars Expressions",{"path":2165,"title":2166,"kind":2143},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fclean-wide-excel-sheets-with-polars-column-selectors","Clean Wide Excel Sheets with Polars Column Selectors",{"path":2168,"title":2169,"kind":2143},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fconvert-pandas-excel-code-to-polars","Convert pandas Excel Code to Polars",{"path":2148,"title":2147,"kind":2171},"fix",{"path":2152,"title":2151,"kind":2143},{"path":2174,"title":2175,"kind":2143},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Fread-multiple-excel-sheets-with-polars","Read Multiple Excel Sheets with Polars",1790972098242]