[{"data":1,"prerenderedAt":1757},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data":1723,"ancestors:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data":1732,"children:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data":1737,"related:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data":1738},{"id":4,"title":5,"body":6,"dateModified":1689,"datePublished":1689,"description":1690,"extension":1691,"faq":1692,"keyword":1701,"kind":1702,"meta":1703,"navigation":99,"path":1712,"seo":1713,"slug":1716,"stem":1717,"topics":1718,"type":1721,"__hash__":1722},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data\u002Findex.md","Turn a Pivot Table Export Back into Tidy Data",{"type":7,"value":8,"toc":1676},"minimark",[9,24,29,58,64,68,71,147,158,162,165,360,375,553,557,560,706,717,721,724,982,1005,1129,1133,1136,1449,1452,1539,1543,1562,1566,1604,1608,1611,1615,1621,1627,1633,1639,1643,1672],[10,11,12,13,17,18,23],"p",{},"You ask for the sales data and receive a pivot table, pasted as values into a new workbook: regions and products down the side, years and quarters across the top in two header rows, a subtotal under each region, a grand total at the bottom and right, and the region name shown only on the first row of its block. It is perfect for reading and useless for analysis — you cannot filter it, join it or re-pivot it a different way. The underlying data can usually be rebuilt: each value cell is one observation of ",[14,15,16],"em",{},"region, product, year, quarter and amount",". This page reverses the pivot layout with pandas step by step, and finishes by recomputing the subtotals and grand total from the rebuilt data and comparing them with the ones in the export — the proof that nothing was lost or double-counted. It belongs to ",[19,20,22],"a",{"href":21},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002F","Extracting Data from Messy Excel Sheets",".",[25,26,28],"h2",{"id":27},"prerequisites","Prerequisites",[30,31,36],"pre",{"className":32,"code":33,"language":34,"meta":35,"style":35},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash","",[37,38,39],"code",{"__ignoreMap":35},[40,41,44,48,52,55],"span",{"class":42,"line":43},"line",1,[40,45,47],{"class":46},"sMTad","pip",[40,49,51],{"class":50},"srMev"," install",[40,53,54],{"class":50}," pandas",[40,56,57],{"class":50}," openpyxl\n",[10,59,60,63],{},[37,61,62],{},"pivot_export.xlsx"," has a title row, a blank row, two header rows — years on the first (each year written once above its quarters), quarters on the second — then three region blocks with products, a bold subtotal row after each region, a grand-total row, and a grand-total column on the right. The region label appears only on the first row of each block, as pivot tables display it.",[25,65,67],{"id":66},"step-1-read-the-sheet-raw","Step 1 — Read the sheet raw",[10,69,70],{},"Do not let pandas guess the header; read everything as a grid and look at it:",[30,72,76],{"className":73,"code":74,"language":75,"meta":35,"style":35},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\nraw = pd.read_excel(\"pivot_export.xlsx\", header=None)\nprint(raw.head(8).to_string())\n","python",[37,77,78,94,101,132],{"__ignoreMap":35},[40,79,80,84,88,91],{"class":42,"line":43},[40,81,83],{"class":82},"s-kum","import",[40,85,87],{"class":86},"skGVy"," pandas ",[40,89,90],{"class":82},"as",[40,92,93],{"class":86}," pd\n",[40,95,97],{"class":42,"line":96},2,[40,98,100],{"emptyLinePlaceholder":99},true,"\n",[40,102,104,107,110,113,116,119,123,125,129],{"class":42,"line":103},3,[40,105,106],{"class":86},"raw ",[40,108,109],{"class":82},"=",[40,111,112],{"class":86}," pd.read_excel(",[40,114,115],{"class":50},"\"pivot_export.xlsx\"",[40,117,118],{"class":86},", ",[40,120,122],{"class":121},"sa561","header",[40,124,109],{"class":82},[40,126,128],{"class":127},"sP0c6","None",[40,130,131],{"class":86},")\n",[40,133,135,138,141,144],{"class":42,"line":134},4,[40,136,137],{"class":127},"print",[40,139,140],{"class":86},"(raw.head(",[40,142,143],{"class":127},"8",[40,145,146],{"class":86},").to_string())\n",[10,148,149,150,153,154,157],{},"With ",[37,151,152],{},"header=None",", every row is data and the columns are numbered. Row 2 holds the years — \"2025\" once, \"2026\" once, with blanks to their right — row 3 the quarter labels and the \"Region\" and \"Product\" captions, and the values start at row 4. Depending on how the pivot was pasted, the year cells may hold numbers or digit-only text, so the code below converts them explicitly with ",[37,155,156],{},"int()"," rather than relying on either. Reading raw first makes every later step explicit about which rows and columns it uses, which matters because layouts like this shift when someone adds a title or a filter row.",[25,159,161],{"id":160},"step-2-rebuild-the-column-headers","Step 2 — Rebuild the column headers",[10,163,164],{},"The value columns are identified by a year and a quarter spread over two header rows, with the year written only once per group. Forward-fill the year row across, then pair it with the quarter row:",[30,166,168],{"className":73,"code":167,"language":75,"meta":35,"style":35},"years = raw.iloc[2].ffill()\nquarters = raw.iloc[3]\ncolumns = []\nfor y, q in zip(years, quarters):\n    if pd.isna(q) or q in (\"Region\", \"Product\"):\n        columns.append(q if pd.notna(q) else \"Total\")\n    else:\n        columns.append(f\"{int(y)} {q}\")\nbody = raw.iloc[4:].copy()\nbody.columns = columns\nprint(columns)\n",[37,169,170,186,201,211,228,259,279,288,325,341,352],{"__ignoreMap":35},[40,171,172,175,177,180,183],{"class":42,"line":43},[40,173,174],{"class":86},"years ",[40,176,109],{"class":82},[40,178,179],{"class":86}," raw.iloc[",[40,181,182],{"class":127},"2",[40,184,185],{"class":86},"].ffill()\n",[40,187,188,191,193,195,198],{"class":42,"line":96},[40,189,190],{"class":86},"quarters ",[40,192,109],{"class":82},[40,194,179],{"class":86},[40,196,197],{"class":127},"3",[40,199,200],{"class":86},"]\n",[40,202,203,206,208],{"class":42,"line":103},[40,204,205],{"class":86},"columns ",[40,207,109],{"class":82},[40,209,210],{"class":86}," []\n",[40,212,213,216,219,222,225],{"class":42,"line":134},[40,214,215],{"class":82},"for",[40,217,218],{"class":86}," y, q ",[40,220,221],{"class":82},"in",[40,223,224],{"class":127}," zip",[40,226,227],{"class":86},"(years, quarters):\n",[40,229,231,234,237,240,243,245,248,251,253,256],{"class":42,"line":230},5,[40,232,233],{"class":82},"    if",[40,235,236],{"class":86}," pd.isna(q) ",[40,238,239],{"class":82},"or",[40,241,242],{"class":86}," q ",[40,244,221],{"class":82},[40,246,247],{"class":86}," (",[40,249,250],{"class":50},"\"Region\"",[40,252,118],{"class":86},[40,254,255],{"class":50},"\"Product\"",[40,257,258],{"class":86},"):\n",[40,260,262,265,268,271,274,277],{"class":42,"line":261},6,[40,263,264],{"class":86},"        columns.append(q ",[40,266,267],{"class":82},"if",[40,269,270],{"class":86}," pd.notna(q) ",[40,272,273],{"class":82},"else",[40,275,276],{"class":50}," \"Total\"",[40,278,131],{"class":86},[40,280,282,285],{"class":42,"line":281},7,[40,283,284],{"class":82},"    else",[40,286,287],{"class":86},":\n",[40,289,291,294,297,300,304,307,310,313,316,319,321,323],{"class":42,"line":290},8,[40,292,293],{"class":86},"        columns.append(",[40,295,296],{"class":82},"f",[40,298,299],{"class":50},"\"",[40,301,303],{"class":302},"sSjpA","{",[40,305,306],{"class":127},"int",[40,308,309],{"class":86},"(y)",[40,311,312],{"class":302},"}",[40,314,315],{"class":302}," {",[40,317,318],{"class":86},"q",[40,320,312],{"class":302},[40,322,299],{"class":50},[40,324,131],{"class":86},[40,326,328,331,333,335,338],{"class":42,"line":327},9,[40,329,330],{"class":86},"body ",[40,332,109],{"class":82},[40,334,179],{"class":86},[40,336,337],{"class":127},"4",[40,339,340],{"class":86},":].copy()\n",[40,342,344,347,349],{"class":42,"line":343},10,[40,345,346],{"class":86},"body.columns ",[40,348,109],{"class":82},[40,350,351],{"class":86}," columns\n",[40,353,355,357],{"class":42,"line":354},11,[40,356,137],{"class":127},[40,358,359],{"class":86},"(columns)\n",[10,361,362,363,366,367,370,371,23],{},"Forward-filling the year row turns ",[37,364,365],{},"2025, NaN, 2026, NaN …"," into ",[37,368,369],{},"2025, 2025, 2026, 2026 …",", so each quarter column knows its year. The \"Region\" and \"Product\" captions become the names of the label columns, and the unlabelled grand-total column is called \"Total\" so it can be dropped deliberately rather than by position. The same approach works for any two-level header; for headers with merged cells, see ",[19,372,374],{"href":373},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fread-excel-with-multi-row-headers-in-pandas\u002F","Read Excel with Multi-Row Headers in pandas",[376,377,385,386,385,390,385,394,385,402,385,411,385,415,385,422,385,426,385,430,385,434,385,438,385,442,385,447,385,449,385,453,385,458,385,460,385,463,385,466,385,469,385,471,385,475,385,477,385,480,385,482,385,485,385,489,385,494,385,499,385,504,385,507,385,511,385,514,385,518,385,524,385,527,385,534,385,539,385,547],"svg",{"viewBox":378,"role":379,"ariaLabelledBy":380,"xmlns":383,"style":384},"47 0 665 168","img",[381,382],"pv-head-t","pv-head-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:665px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[387,388,389],"title",{"id":381},"Two header rows become one name per column",[391,392,393],"desc",{"id":382},"The year row has 2025 above two quarters and 2026 above four, written once each. After forward-filling the years and combining them with the quarter row, the value columns are named 2025 Q3, 2025 Q4, 2026 Q1 and so on.",[395,396],"rect",{"x":397,"y":398,"width":399,"height":400,"fill":401},"47","0","665","168","#ffffff",[395,403],{"x":404,"y":405,"width":406,"height":407,"rx":398,"fill":408,"stroke":409,"style":410},"63.5","40.0","32.0","24.0","#e7ebef","var(--line,#cdd5e6)","stroke-width:1px",[395,412],{"x":413,"y":405,"width":414,"height":407,"rx":398,"fill":408,"stroke":409,"style":410},"95.5","52.2",[416,417,421],"text",{"x":418,"y":419,"style":420},"121.6","57.0","font-size:11px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","A",[395,423],{"x":424,"y":405,"width":425,"height":407,"rx":398,"fill":408,"stroke":409,"style":410},"147.7","44.0",[416,427,429],{"x":428,"y":419,"style":420},"169.7","B",[395,431],{"x":432,"y":405,"width":433,"height":407,"rx":398,"fill":408,"stroke":409,"style":410},"191.7","52.5",[416,435,437],{"x":436,"y":419,"style":420},"217.9","C",[395,439],{"x":404,"y":440,"width":406,"height":441,"rx":398,"fill":408,"stroke":409,"style":410},"64.0","28.0",[416,443,446],{"x":444,"y":445,"style":420},"79.5","83.0","1",[395,448],{"x":413,"y":440,"width":414,"height":441,"rx":398,"fill":401,"stroke":409,"style":410},[416,450,452],{"x":418,"y":445,"style":451},"font-size:10.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","2025",[395,454],{"x":424,"y":440,"width":425,"height":441,"rx":398,"fill":455,"stroke":456,"style":457},"#fce9e9","var(--danger,#dc2626)","stroke-width:1.5px",[395,459],{"x":432,"y":440,"width":433,"height":441,"rx":398,"fill":401,"stroke":409,"style":410},[416,461,462],{"x":436,"y":445,"style":451},"2026",[395,464],{"x":404,"y":465,"width":406,"height":441,"rx":398,"fill":408,"stroke":409,"style":410},"92.0",[416,467,182],{"x":444,"y":468,"style":420},"111.0",[395,470],{"x":413,"y":465,"width":414,"height":441,"rx":398,"fill":401,"stroke":409,"style":410},[416,472,474],{"x":418,"y":468,"style":473},"font-size:10.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","Q3",[395,476],{"x":424,"y":465,"width":425,"height":441,"rx":398,"fill":401,"stroke":409,"style":410},[416,478,479],{"x":428,"y":468,"style":473},"Q4",[395,481],{"x":432,"y":465,"width":433,"height":441,"rx":398,"fill":401,"stroke":409,"style":410},[416,483,484],{"x":436,"y":468,"style":473},"Q1",[395,486],{"x":487,"y":405,"width":488,"height":441,"rx":398,"fill":408,"stroke":409,"style":457},"599.9","96.6",[416,490,493],{"x":491,"y":492,"style":451},"648.2","59.0","column name",[395,495],{"x":487,"y":496,"width":488,"height":441,"rx":398,"fill":497,"stroke":498,"style":457},"68.0","#d9f4f1","var(--teal,#0f9488)",[416,500,503],{"x":491,"y":501,"style":502},"87.0","font-size:10.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","2025 Q3",[395,505],{"x":487,"y":506,"width":488,"height":441,"rx":398,"fill":497,"stroke":498,"style":457},"96.0",[416,508,510],{"x":491,"y":509,"style":502},"115.0","2025 Q4",[395,512],{"x":487,"y":513,"width":488,"height":441,"rx":398,"fill":497,"stroke":498,"style":457},"124.0",[416,515,517],{"x":491,"y":516,"style":502},"143.0","2026 Q1",[416,519,523],{"x":520,"y":521,"style":522},"153.8","29.0","font-size:12.5px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","header rows",[416,525,526],{"x":491,"y":521,"style":522},"column names",[42,528],{"x1":529,"y1":530,"x2":531,"y2":530,"stroke":532,"style":533},"254.2","80.0","585.9","var(--brand,#5b5cf0)","stroke-width:2px",[535,536],"polygon",{"points":537,"fill":538},"589.9,80.0 580.9,85.0 580.9,75.0","#5b5cf0",[395,540],{"x":541,"y":542,"width":543,"height":544,"rx":545,"fill":401,"stroke":409,"style":546},"295.7","47.0","252.6","22.0","11","stroke-width:1.2px",[416,548,552],{"x":549,"y":550,"style":551},"422.1","62.0","font-size:11.5px;font-weight:400;fill:var(--brand-strong,#4338ca);text-anchor:middle;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace","ffill years + pair with quarters",[25,554,556],{"id":555},"step-3-fill-the-row-labels-and-drop-the-total-rows","Step 3 — Fill the row labels and drop the total rows",[10,558,559],{},"Pivot layouts show a region once per block. Forward-fill it down, then remove the subtotal and grand-total rows — but keep them aside, because they are the check in Step 5:",[30,561,563],{"className":73,"code":562,"language":75,"meta":35,"style":35},"body[\"Region\"] = body[\"Region\"].ffill()\nis_total = body[\"Region\"].astype(str).str.contains(r\"Total$\", regex=True)\ntotals = body[is_total].copy()\ndetail = body[~is_total & body[\"Product\"].notna()].copy()\nprint(detail[[\"Region\", \"Product\"]].to_string(index=False))\nprint(\"total rows set aside:\", totals[\"Region\"].tolist())\n",[37,564,565,584,627,637,661,688],{"__ignoreMap":35},[40,566,567,570,572,575,577,580,582],{"class":42,"line":43},[40,568,569],{"class":86},"body[",[40,571,250],{"class":50},[40,573,574],{"class":86},"] ",[40,576,109],{"class":82},[40,578,579],{"class":86}," body[",[40,581,250],{"class":50},[40,583,185],{"class":86},[40,585,586,589,591,593,595,598,601,604,607,610,613,615,617,620,622,625],{"class":42,"line":96},[40,587,588],{"class":86},"is_total ",[40,590,109],{"class":82},[40,592,579],{"class":86},[40,594,250],{"class":50},[40,596,597],{"class":86},"].astype(",[40,599,600],{"class":127},"str",[40,602,603],{"class":86},").str.contains(",[40,605,606],{"class":82},"r",[40,608,609],{"class":50},"\"Total",[40,611,612],{"class":127},"$",[40,614,299],{"class":50},[40,616,118],{"class":86},[40,618,619],{"class":121},"regex",[40,621,109],{"class":82},[40,623,624],{"class":127},"True",[40,626,131],{"class":86},[40,628,629,632,634],{"class":42,"line":103},[40,630,631],{"class":86},"totals ",[40,633,109],{"class":82},[40,635,636],{"class":86}," body[is_total].copy()\n",[40,638,639,642,644,646,649,651,654,656,658],{"class":42,"line":134},[40,640,641],{"class":86},"detail ",[40,643,109],{"class":82},[40,645,579],{"class":86},[40,647,648],{"class":82},"~",[40,650,588],{"class":86},[40,652,653],{"class":82},"&",[40,655,579],{"class":86},[40,657,255],{"class":50},[40,659,660],{"class":86},"].notna()].copy()\n",[40,662,663,665,668,670,672,674,677,680,682,685],{"class":42,"line":230},[40,664,137],{"class":127},[40,666,667],{"class":86},"(detail[[",[40,669,250],{"class":50},[40,671,118],{"class":86},[40,673,255],{"class":50},[40,675,676],{"class":86},"]].to_string(",[40,678,679],{"class":121},"index",[40,681,109],{"class":82},[40,683,684],{"class":127},"False",[40,686,687],{"class":86},"))\n",[40,689,690,692,695,698,701,703],{"class":42,"line":261},[40,691,137],{"class":127},[40,693,694],{"class":86},"(",[40,696,697],{"class":50},"\"total rows set aside:\"",[40,699,700],{"class":86},", totals[",[40,702,250],{"class":50},[40,704,705],{"class":86},"].tolist())\n",[10,707,708,709,712,713,23],{},"After the forward fill, every product row carries its region. Subtotal rows are recognised by their label ending in \"Total\" — \"North Total\", \"Grand Total\" — the default wording of Excel pivot tables; if your export uses another language or custom labels, adjust the pattern. The extra condition ",[37,710,711],{},"Product.notna()"," drops any row with no product, which also catches total rows whose label was missed. Six detail rows remain: the real data. Removing subtotal rows from ordinary reports is covered in ",[19,714,716],{"href":715},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fremove-subtotal-and-total-rows-from-an-excel-export\u002F","Remove Subtotal and Total Rows from an Excel Export",[25,718,720],{"id":719},"step-4-unpivot-into-one-row-per-observation","Step 4 — Unpivot into one row per observation",[10,722,723],{},"Now melt the quarter columns into rows, drop the grand-total column, and split the column names back into year and quarter:",[30,725,727],{"className":73,"code":726,"language":75,"meta":35,"style":35},"value_cols = [c for c in detail.columns if c not in (\"Region\", \"Product\", \"Total\")]\ntidy = detail.melt(id_vars=[\"Region\", \"Product\"], value_vars=value_cols, var_name=\"Period\", value_name=\"Units\")\ntidy[[\"Year\", \"Quarter\"]] = tidy[\"Period\"].str.split(\" \", expand=True)\ntidy[\"Year\"] = tidy[\"Year\"].astype(int)\ntidy[\"Units\"] = pd.to_numeric(tidy[\"Units\"])\ntidy = tidy.drop(columns=\"Period\").sort_values([\"Region\", \"Product\", \"Year\", \"Quarter\"]).reset_index(drop=True)\nprint(tidy.head(6).to_string(index=False))\nprint(tidy.shape)\n",[37,728,729,775,830,870,891,909,954,975],{"__ignoreMap":35},[40,730,731,734,736,739,741,744,746,749,751,753,756,759,761,763,765,767,769,772],{"class":42,"line":43},[40,732,733],{"class":86},"value_cols ",[40,735,109],{"class":82},[40,737,738],{"class":86}," [c ",[40,740,215],{"class":82},[40,742,743],{"class":86}," c ",[40,745,221],{"class":82},[40,747,748],{"class":86}," detail.columns ",[40,750,267],{"class":82},[40,752,743],{"class":86},[40,754,755],{"class":82},"not",[40,757,758],{"class":82}," in",[40,760,247],{"class":86},[40,762,250],{"class":50},[40,764,118],{"class":86},[40,766,255],{"class":50},[40,768,118],{"class":86},[40,770,771],{"class":50},"\"Total\"",[40,773,774],{"class":86},")]\n",[40,776,777,780,782,785,788,790,793,795,797,799,802,805,807,810,813,815,818,820,823,825,828],{"class":42,"line":96},[40,778,779],{"class":86},"tidy ",[40,781,109],{"class":82},[40,783,784],{"class":86}," detail.melt(",[40,786,787],{"class":121},"id_vars",[40,789,109],{"class":82},[40,791,792],{"class":86},"[",[40,794,250],{"class":50},[40,796,118],{"class":86},[40,798,255],{"class":50},[40,800,801],{"class":86},"], ",[40,803,804],{"class":121},"value_vars",[40,806,109],{"class":82},[40,808,809],{"class":86},"value_cols, ",[40,811,812],{"class":121},"var_name",[40,814,109],{"class":82},[40,816,817],{"class":50},"\"Period\"",[40,819,118],{"class":86},[40,821,822],{"class":121},"value_name",[40,824,109],{"class":82},[40,826,827],{"class":50},"\"Units\"",[40,829,131],{"class":86},[40,831,832,835,838,840,843,846,848,851,853,856,859,861,864,866,868],{"class":42,"line":103},[40,833,834],{"class":86},"tidy[[",[40,836,837],{"class":50},"\"Year\"",[40,839,118],{"class":86},[40,841,842],{"class":50},"\"Quarter\"",[40,844,845],{"class":86},"]] ",[40,847,109],{"class":82},[40,849,850],{"class":86}," tidy[",[40,852,817],{"class":50},[40,854,855],{"class":86},"].str.split(",[40,857,858],{"class":50},"\" \"",[40,860,118],{"class":86},[40,862,863],{"class":121},"expand",[40,865,109],{"class":82},[40,867,624],{"class":127},[40,869,131],{"class":86},[40,871,872,875,877,879,881,883,885,887,889],{"class":42,"line":134},[40,873,874],{"class":86},"tidy[",[40,876,837],{"class":50},[40,878,574],{"class":86},[40,880,109],{"class":82},[40,882,850],{"class":86},[40,884,837],{"class":50},[40,886,597],{"class":86},[40,888,306],{"class":127},[40,890,131],{"class":86},[40,892,893,895,897,899,901,904,906],{"class":42,"line":230},[40,894,874],{"class":86},[40,896,827],{"class":50},[40,898,574],{"class":86},[40,900,109],{"class":82},[40,902,903],{"class":86}," pd.to_numeric(tidy[",[40,905,827],{"class":50},[40,907,908],{"class":86},"])\n",[40,910,911,913,915,918,921,923,925,928,930,932,934,936,938,940,942,945,948,950,952],{"class":42,"line":261},[40,912,779],{"class":86},[40,914,109],{"class":82},[40,916,917],{"class":86}," tidy.drop(",[40,919,920],{"class":121},"columns",[40,922,109],{"class":82},[40,924,817],{"class":50},[40,926,927],{"class":86},").sort_values([",[40,929,250],{"class":50},[40,931,118],{"class":86},[40,933,255],{"class":50},[40,935,118],{"class":86},[40,937,837],{"class":50},[40,939,118],{"class":86},[40,941,842],{"class":50},[40,943,944],{"class":86},"]).reset_index(",[40,946,947],{"class":121},"drop",[40,949,109],{"class":82},[40,951,624],{"class":127},[40,953,131],{"class":86},[40,955,956,958,961,964,967,969,971,973],{"class":42,"line":281},[40,957,137],{"class":127},[40,959,960],{"class":86},"(tidy.head(",[40,962,963],{"class":127},"6",[40,965,966],{"class":86},").to_string(",[40,968,679],{"class":121},[40,970,109],{"class":82},[40,972,684],{"class":127},[40,974,687],{"class":86},[40,976,977,979],{"class":42,"line":290},[40,978,137],{"class":127},[40,980,981],{"class":86},"(tidy.shape)\n",[10,983,984,985,988,989,993,994,997,998,1001,1002,23],{},"Six product rows times six quarters give 36 observations, each with region, product, year, quarter and units — the shape the original data had before it was pivoted. ",[37,986,987],{},"pd.to_numeric"," makes sure the units are numbers; if the export had formatted values as text with thousands separators, convert them first as in ",[19,990,992],{"href":991},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fconvert-excel-text-columns-to-numbers-with-pandas\u002F","Convert Excel Text Columns to Numbers with pandas",". Blank cells in a pivot usually mean \"no data\" rather than zero; after ",[37,995,996],{},"melt"," they become ",[37,999,1000],{},"NaN"," rows that you can drop with ",[37,1003,1004],{},"dropna(subset=[\"Units\"])",[376,1006,385,1012,385,1015,385,1018,385,1022,385,1029,385,1040,385,1046,385,1050,385,1055,385,1058,385,1062,385,1069,385,1074,385,1077,385,1081,385,1084,385,1089,385,1096,385,1101,385,1104,385,1108,385,1111,385,1114,385,1121,385,1126],{"viewBox":1007,"role":379,"ariaLabelledBy":1008,"xmlns":383,"style":1011},"4 0 752 125",[1009,1010],"pv-steps-t","pv-steps-d","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[387,1013,1014],{"id":1009},"Reversing a pivot layout",[391,1016,1017],{"id":1010},"Read the sheet without headers, combine the two header rows into column names, forward-fill the row labels, set the total rows aside, unpivot the value columns into one row per observation, then recompute the totals and compare them with the ones that were set aside.",[395,1019],{"x":337,"y":398,"width":1020,"height":1021,"fill":401},"752","125",[395,1023],{"x":1024,"y":1025,"width":1026,"height":1027,"rx":1028,"fill":408,"stroke":409,"style":533},"20.0","16.0","153.0","93.0","12",[1030,1031,1033],"g",{"transform":1032},"translate(84.5,29.0) scale(1.000)",[1034,1035],"path",{"d":1036,"fill":1037,"stroke":1038,"style":1039},"M4 4h16v16H4z M4 9h16 M4 14h16 M10 4v16","none","var(--text,#172033)","stroke-width:1.80px;stroke-linecap:round;stroke-linejoin:round",[416,1041,1045],{"x":1042,"y":1043,"style":1044},"96.5","75.5","font-size:13.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","Read raw",[416,1047,152],{"x":1042,"y":1048,"style":1049},"95.0","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle",[42,1051],{"x1":1052,"y1":1053,"x2":1054,"y2":1053,"stroke":532,"style":533},"178.0","62.5","200.0",[535,1056],{"points":1057,"fill":538},"204.0,62.5 195.0,67.5 195.0,57.5",[395,1059],{"x":1060,"y":1025,"width":1026,"height":1027,"rx":1028,"fill":1061,"stroke":532,"style":533},"209.0","#f0f4ff",[1030,1063,1065],{"transform":1064},"translate(273.5,29.0) scale(1.000)",[1034,1066],{"d":1067,"fill":1037,"stroke":1068,"style":1039},"M3 5h18v14H3z M3 10h18 M3 15h18 M9 5v14","var(--brand-strong,#4338ca)",[416,1070,1073],{"x":1071,"y":1043,"style":1072},"285.5","font-size:13.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Rebuild headers",[416,1075,1076],{"x":1071,"y":1048,"style":1049},"ffill years + quarters",[42,1078],{"x1":1079,"y1":1053,"x2":1080,"y2":1053,"stroke":532,"style":533},"367.0","389.0",[535,1082],{"points":1083,"fill":538},"393.0,62.5 384.0,67.5 384.0,57.5",[395,1085],{"x":1086,"y":1025,"width":1026,"height":1027,"rx":1028,"fill":1087,"stroke":1088,"style":533},"398.0","#fdefd8","var(--gold,#b4740a)",[1030,1090,1092],{"transform":1091},"translate(462.5,29.0) scale(1.000)",[1034,1093],{"d":1094,"fill":1037,"stroke":1095,"style":1039},"M3 5h18l-7 8v6l-4 2v-8z","var(--gold-ink,#7a4e06)",[416,1097,1100],{"x":1098,"y":1043,"style":1099},"474.5","font-size:13.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","Labels + totals",[416,1102,1103],{"x":1098,"y":1048,"style":1049},"ffill, set totals aside",[42,1105],{"x1":1106,"y1":1053,"x2":1107,"y2":1053,"stroke":532,"style":533},"556.0","578.0",[535,1109],{"points":1110,"fill":538},"582.0,62.5 573.0,67.5 573.0,57.5",[395,1112],{"x":1113,"y":1025,"width":1026,"height":1027,"rx":1028,"fill":497,"stroke":498,"style":533},"587.0",[1030,1115,1117],{"transform":1116},"translate(651.5,29.0) scale(1.000)",[1034,1118],{"d":1119,"fill":1037,"stroke":1120,"style":1039},"M12 3l9 5-9 5-9-5z M3 13l9 5 9-5 M3 17l9 5 9-5","var(--teal-ink,#0b6157)",[416,1122,1125],{"x":1123,"y":1043,"style":1124},"663.5","font-size:13.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","Melt",[416,1127,1128],{"x":1123,"y":1048,"style":1049},"one row per value",[25,1130,1132],{"id":1131},"step-5-prove-it-by-recomputing-the-totals","Step 5 — Prove it by recomputing the totals",[10,1134,1135],{},"The export contained its own checksums: the subtotals, the grand-total row and the grand-total column. Recompute them from the tidy data and compare:",[30,1137,1139],{"className":73,"code":1138,"language":75,"meta":35,"style":35},"tidy[\"Period\"] = tidy[\"Year\"].astype(str) + \" \" + tidy[\"Quarter\"]\nrecomputed = tidy.pivot_table(index=\"Region\", columns=\"Period\", values=\"Units\", aggfunc=\"sum\")\nrecomputed = recomputed[value_cols]\n\nexpected = totals.set_index(\"Region\")[value_cols].astype(int)\nexpected.index = expected.index.str.replace(\" Total\", \"\", regex=False)\ngrand_expected = expected.loc[\"Grand\"]\nregional_ok = (recomputed.loc[[\"North\", \"South\", \"West\"]].astype(int).values == expected.drop(\"Grand\").values).all()\ngrand_ok = (recomputed.sum().astype(int).values == grand_expected.values).all()\nrow_totals_ok = (detail.set_index([\"Region\", \"Product\"])[\"Total\"].astype(int)\n                 == tidy.groupby([\"Region\", \"Product\"])[\"Units\"].sum()).all()\nprint(\"regional subtotals match:\", regional_ok, \"| grand total row matches:\", grand_ok,\n      \"| row totals match:\", row_totals_ok, \"| grand total:\", int(tidy[\"Units\"].sum()))\n",[37,1140,1141,1177,1222,1231,1235,1254,1282,1297,1339,1358,1385,1406,1425],{"__ignoreMap":35},[40,1142,1143,1145,1147,1149,1151,1153,1155,1157,1159,1162,1165,1168,1171,1173,1175],{"class":42,"line":43},[40,1144,874],{"class":86},[40,1146,817],{"class":50},[40,1148,574],{"class":86},[40,1150,109],{"class":82},[40,1152,850],{"class":86},[40,1154,837],{"class":50},[40,1156,597],{"class":86},[40,1158,600],{"class":127},[40,1160,1161],{"class":86},") ",[40,1163,1164],{"class":82},"+",[40,1166,1167],{"class":50}," \" \"",[40,1169,1170],{"class":82}," +",[40,1172,850],{"class":86},[40,1174,842],{"class":50},[40,1176,200],{"class":86},[40,1178,1179,1182,1184,1187,1189,1191,1193,1195,1197,1199,1201,1203,1206,1208,1210,1212,1215,1217,1220],{"class":42,"line":96},[40,1180,1181],{"class":86},"recomputed ",[40,1183,109],{"class":82},[40,1185,1186],{"class":86}," tidy.pivot_table(",[40,1188,679],{"class":121},[40,1190,109],{"class":82},[40,1192,250],{"class":50},[40,1194,118],{"class":86},[40,1196,920],{"class":121},[40,1198,109],{"class":82},[40,1200,817],{"class":50},[40,1202,118],{"class":86},[40,1204,1205],{"class":121},"values",[40,1207,109],{"class":82},[40,1209,827],{"class":50},[40,1211,118],{"class":86},[40,1213,1214],{"class":121},"aggfunc",[40,1216,109],{"class":82},[40,1218,1219],{"class":50},"\"sum\"",[40,1221,131],{"class":86},[40,1223,1224,1226,1228],{"class":42,"line":103},[40,1225,1181],{"class":86},[40,1227,109],{"class":82},[40,1229,1230],{"class":86}," recomputed[value_cols]\n",[40,1232,1233],{"class":42,"line":134},[40,1234,100],{"emptyLinePlaceholder":99},[40,1236,1237,1240,1242,1245,1247,1250,1252],{"class":42,"line":230},[40,1238,1239],{"class":86},"expected ",[40,1241,109],{"class":82},[40,1243,1244],{"class":86}," totals.set_index(",[40,1246,250],{"class":50},[40,1248,1249],{"class":86},")[value_cols].astype(",[40,1251,306],{"class":127},[40,1253,131],{"class":86},[40,1255,1256,1259,1261,1264,1267,1269,1272,1274,1276,1278,1280],{"class":42,"line":261},[40,1257,1258],{"class":86},"expected.index ",[40,1260,109],{"class":82},[40,1262,1263],{"class":86}," expected.index.str.replace(",[40,1265,1266],{"class":50},"\" Total\"",[40,1268,118],{"class":86},[40,1270,1271],{"class":50},"\"\"",[40,1273,118],{"class":86},[40,1275,619],{"class":121},[40,1277,109],{"class":82},[40,1279,684],{"class":127},[40,1281,131],{"class":86},[40,1283,1284,1287,1289,1292,1295],{"class":42,"line":281},[40,1285,1286],{"class":86},"grand_expected ",[40,1288,109],{"class":82},[40,1290,1291],{"class":86}," expected.loc[",[40,1293,1294],{"class":50},"\"Grand\"",[40,1296,200],{"class":86},[40,1298,1299,1302,1304,1307,1310,1312,1315,1317,1320,1323,1325,1328,1331,1334,1336],{"class":42,"line":290},[40,1300,1301],{"class":86},"regional_ok ",[40,1303,109],{"class":82},[40,1305,1306],{"class":86}," (recomputed.loc[[",[40,1308,1309],{"class":50},"\"North\"",[40,1311,118],{"class":86},[40,1313,1314],{"class":50},"\"South\"",[40,1316,118],{"class":86},[40,1318,1319],{"class":50},"\"West\"",[40,1321,1322],{"class":86},"]].astype(",[40,1324,306],{"class":127},[40,1326,1327],{"class":86},").values ",[40,1329,1330],{"class":82},"==",[40,1332,1333],{"class":86}," expected.drop(",[40,1335,1294],{"class":50},[40,1337,1338],{"class":86},").values).all()\n",[40,1340,1341,1344,1346,1349,1351,1353,1355],{"class":42,"line":327},[40,1342,1343],{"class":86},"grand_ok ",[40,1345,109],{"class":82},[40,1347,1348],{"class":86}," (recomputed.sum().astype(",[40,1350,306],{"class":127},[40,1352,1327],{"class":86},[40,1354,1330],{"class":82},[40,1356,1357],{"class":86}," grand_expected.values).all()\n",[40,1359,1360,1363,1365,1368,1370,1372,1374,1377,1379,1381,1383],{"class":42,"line":343},[40,1361,1362],{"class":86},"row_totals_ok ",[40,1364,109],{"class":82},[40,1366,1367],{"class":86}," (detail.set_index([",[40,1369,250],{"class":50},[40,1371,118],{"class":86},[40,1373,255],{"class":50},[40,1375,1376],{"class":86},"])[",[40,1378,771],{"class":50},[40,1380,597],{"class":86},[40,1382,306],{"class":127},[40,1384,131],{"class":86},[40,1386,1387,1390,1393,1395,1397,1399,1401,1403],{"class":42,"line":354},[40,1388,1389],{"class":82},"                 ==",[40,1391,1392],{"class":86}," tidy.groupby([",[40,1394,250],{"class":50},[40,1396,118],{"class":86},[40,1398,255],{"class":50},[40,1400,1376],{"class":86},[40,1402,827],{"class":50},[40,1404,1405],{"class":86},"].sum()).all()\n",[40,1407,1409,1411,1413,1416,1419,1422],{"class":42,"line":1408},12,[40,1410,137],{"class":127},[40,1412,694],{"class":86},[40,1414,1415],{"class":50},"\"regional subtotals match:\"",[40,1417,1418],{"class":86},", regional_ok, ",[40,1420,1421],{"class":50},"\"| grand total row matches:\"",[40,1423,1424],{"class":86},", grand_ok,\n",[40,1426,1428,1431,1434,1437,1439,1441,1444,1446],{"class":42,"line":1427},13,[40,1429,1430],{"class":50},"      \"| row totals match:\"",[40,1432,1433],{"class":86},", row_totals_ok, ",[40,1435,1436],{"class":50},"\"| grand total:\"",[40,1438,118],{"class":86},[40,1440,306],{"class":127},[40,1442,1443],{"class":86},"(tidy[",[40,1445,827],{"class":50},[40,1447,1448],{"class":86},"].sum()))\n",[10,1450,1451],{},"Every regional subtotal, the grand-total row and each product's row total in the export equal the figures recomputed from the 36 tidy rows. That is strong evidence the rebuild is complete and correct: a missed row, a duplicated row or a misread header would break at least one of these sums. If a check fails, the export may have been filtered (pivot tables can hide items) or the subtotal labels may not be what you expected — both worth asking the sender about.",[376,1453,385,1458,385,1461,385,1464,385,1467,385,1470,385,1474,385,1480,385,1483,385,1487,385,1491,385,1496,385,1498,385,1502,385,1504,385,1507,385,1510,385,1514,385,1516,385,1519,385,1521,385,1523,385,1525,385,1529,385,1531,385,1535,385,1537],{"viewBox":1454,"role":379,"ariaLabelledBy":1455,"xmlns":383,"style":1011},"4 0 752 174",[1456,1457],"pv-checks-t","pv-checks-d",[387,1459,1460],{"id":1456},"The export's own totals as checks",[391,1462,1463],{"id":1457},"Regional subtotal rows are compared with sums by region, the grand total row with sums by period, and the grand total column with sums by region and product. All three matched for the rebuilt data.",[395,1465],{"x":337,"y":398,"width":1020,"height":1466,"fill":401},"174",[395,1468],{"x":1024,"y":1025,"width":1469,"height":405,"rx":398,"fill":401,"stroke":409,"style":410},"160.0",[395,1471],{"x":1472,"y":1025,"width":1473,"height":405,"rx":398,"fill":1061,"stroke":532,"style":546},"180.0","280.0",[416,1475,1479],{"x":1476,"y":1477,"style":1478},"320.0","41.0","font-size:13px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","compare with",[395,1481],{"x":1482,"y":1025,"width":1473,"height":405,"rx":398,"fill":1061,"stroke":532,"style":546},"460.0",[416,1484,1486],{"x":1485,"y":1477,"style":1478},"600.0","result",[395,1488],{"x":1024,"y":1489,"width":1469,"height":1490,"rx":398,"fill":408,"stroke":409,"style":410},"56.0","34.0",[416,1492,1495],{"x":406,"y":1493,"style":1494},"78.0","font-size:12.5px;font-weight:700;fill:var(--text,#172033);text-anchor:start","region subtotal rows",[395,1497],{"x":1472,"y":1489,"width":1473,"height":1490,"rx":398,"fill":401,"stroke":409,"style":410},[416,1499,1501],{"x":1476,"y":1500,"style":1049},"77.0","sum by region and period",[395,1503],{"x":1482,"y":1489,"width":1473,"height":1490,"rx":398,"fill":401,"stroke":409,"style":410},[416,1505,1506],{"x":1485,"y":1500,"style":1049},"match",[395,1508],{"x":1024,"y":1509,"width":1469,"height":1490,"rx":398,"fill":408,"stroke":409,"style":410},"90.0",[416,1511,1513],{"x":406,"y":1512,"style":1494},"112.0","grand total row",[395,1515],{"x":1472,"y":1509,"width":1473,"height":1490,"rx":398,"fill":401,"stroke":409,"style":410},[416,1517,1518],{"x":1476,"y":468,"style":1049},"sum by period",[395,1520],{"x":1482,"y":1509,"width":1473,"height":1490,"rx":398,"fill":401,"stroke":409,"style":410},[416,1522,1506],{"x":1485,"y":468,"style":1049},[395,1524],{"x":1024,"y":513,"width":1469,"height":1490,"rx":398,"fill":408,"stroke":409,"style":410},[416,1526,1528],{"x":406,"y":1527,"style":1494},"146.0","grand total column",[395,1530],{"x":1472,"y":513,"width":1473,"height":1490,"rx":398,"fill":401,"stroke":409,"style":410},[416,1532,1534],{"x":1476,"y":1533,"style":1049},"145.0","sum by region and product",[395,1536],{"x":1482,"y":513,"width":1473,"height":1490,"rx":398,"fill":401,"stroke":409,"style":410},[416,1538,1506],{"x":1485,"y":1533,"style":1049},[25,1540,1542],{"id":1541},"variations-you-will-meet","Variations you will meet",[10,1544,1545,1546,1549,1550,1553,1554,1557,1558,1561],{},"Pivot exports come in a few other shapes, and the same steps adapt. With ",[14,1547,1548],{},"several value fields"," — \"Sum of Units\" and \"Sum of Revenue\" side by side under each quarter — there is a third header level; forward-fill it like the year row, build names such as ",[37,1551,1552],{},"2026 Q1|Units",", and after melting split the name into period and measure, then pivot the measure back into columns so units and revenue sit side by side on each row. With the ",[14,1555,1556],{},"compact layout",", where regions and products share one indented column, the region rows have no values and the product rows are indented; detect region rows by their empty value cells (or by the cell's indent, readable with openpyxl), forward-fill them into a new column, and drop them. And with ",[14,1559,1560],{},"row grand totals switched off",", simply skip the row-total check. In every case, the export's remaining totals are the test of the rebuild.",[25,1563,1565],{"id":1564},"common-pitfalls","Common pitfalls",[1567,1568,1569,1580,1586,1592,1598],"ul",{},[1570,1571,1572,1576,1577,1579],"li",{},[1573,1574,1575],"strong",{},"Letting pandas pick the header."," Read with ",[37,1578,152],{}," and build headers explicitly.",[1570,1581,1582,1585],{},[1573,1583,1584],{},"Forgetting the blank labels."," Forward-fill region labels before filtering.",[1570,1587,1588,1591],{},[1573,1589,1590],{},"Keeping subtotal rows."," They double-count; identify them by their labels and set them aside.",[1570,1593,1594,1597],{},[1573,1595,1596],{},"Year cells that are sometimes numbers, sometimes text."," Convert them explicitly.",[1570,1599,1600,1603],{},[1573,1601,1602],{},"Skipping the total check."," Recompute the export's totals to prove nothing was lost.",[25,1605,1607],{"id":1606},"conclusion","Conclusion",[10,1609,1610],{},"A pasted pivot table can be turned back into data: read it without headers, forward-fill the year row and pair it with the quarter row to name each value column, forward-fill the row labels pivot layouts leave blank, set the subtotal and grand-total rows aside, and melt the value columns into one row per observation. Then recompute the subtotals and grand totals from the tidy data and compare them with the ones in the export — if they all match, the rebuild is complete. And next time, ask for the source data.",[25,1612,1614],{"id":1613},"frequently-asked-questions","Frequently asked questions",[10,1616,1617,1620],{},[1573,1618,1619],{},"How do I unpivot an Excel pivot table in Python?","\nRead it with header=None, build column names from the header rows, forward-fill the row labels, drop subtotal and grand total rows, and use pandas melt to turn the value columns into rows.",[10,1622,1623,1626],{},[1573,1624,1625],{},"How do I handle the blank row labels in a pivot table export?","\nForward-fill the label column with ffill so every row carries the label shown on the first row of its block.",[10,1628,1629,1632],{},[1573,1630,1631],{},"How do I remove subtotal rows from a pivot export?","\nIdentify them by their labels, such as names ending in Total, keep them in a separate frame for checking, and filter them out of the data.",[10,1634,1635,1638],{},[1573,1636,1637],{},"How can I check that the unpivoted data is correct?","\nRecompute the subtotals and grand totals from the tidy data with a pivot table and compare them with the totals that were in the export.",[25,1640,1642],{"id":1641},"related","Related",[1567,1644,1645,1651,1658,1665],{},[1570,1646,1647,1648,1650],{},"Up one level: ",[19,1649,22],{"href":21}," — getting data out of awkward layouts.",[1570,1652,1653,1657],{},[19,1654,1656],{"href":1655},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcreating-pivot-tables-from-excel-data\u002Funpivot-a-wide-excel-sheet-with-pandas-melt\u002F","Unpivot a Wide Excel Sheet with pandas melt"," — the melt step on its own.",[1570,1659,1660,1664],{},[19,1661,1663],{"href":1662},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fforward-fill-blank-cells-in-grouped-excel-reports\u002F","Forward-Fill Blank Cells in Grouped Excel Reports"," — filling labels in other report layouts.",[1570,1666,1667,1671],{},[19,1668,1670],{"href":1669},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcreating-pivot-tables-from-excel-data\u002Fcreate-pivot-table-from-excel-with-pandas\u002F","Create a Pivot Table from Excel with pandas"," — going the other way.",[1673,1674,1675],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":35,"searchDepth":96,"depth":96,"links":1677},[1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688],{"id":27,"depth":96,"text":28},{"id":66,"depth":96,"text":67},{"id":160,"depth":96,"text":161},{"id":555,"depth":96,"text":556},{"id":719,"depth":96,"text":720},{"id":1131,"depth":96,"text":1132},{"id":1541,"depth":96,"text":1542},{"id":1564,"depth":96,"text":1565},{"id":1606,"depth":96,"text":1607},{"id":1613,"depth":96,"text":1614},{"id":1641,"depth":96,"text":1642},"2026-10-02","Someone sends a copy-pasted pivot table instead of the data behind it. Rebuild one row per observation with pandas: combine the two header rows into year and quarter, forward-fill the row labels that pivot layouts leave blank, drop subtotal and grand-total rows, unpivot the value columns, convert the numbers, and prove the result by recomputing the totals the export showed.","md",[1693,1695,1697,1699],{"q":1619,"a":1694},"Read it with header=None, build column names from the header rows, forward-fill the row labels, drop subtotal and grand total rows, and use pandas melt to turn the value columns into rows.",{"q":1625,"a":1696},"Forward-fill the label column with ffill so every row carries the label shown on the first row of its block.",{"q":1631,"a":1698},"Identify them by their labels, such as names ending in Total, keep them in a separate frame for checking, and filter them out of the data.",{"q":1637,"a":1700},"Recompute the subtotals and grand totals from the tidy data with a pivot table and compare them with the totals that were in the export.","turn a pivot table export back into tidy data",null,{"breadcrumb":1704},[1705,1708,1711],{"name":1706,"item":1707},"Home","\u002F",{"name":1709,"item":1710},"Advanced Data Transformation and Cleaning for Excel Automation","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":22,"item":21},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data",{"title":1714,"description":1715},"Turn an Excel Pivot Table Export Back into Tidy Data","Reverse a pasted pivot table with pandas: two-row headers into columns, forward-fill blank row labels, drop subtotal and grand total rows, melt to long format, and check against the original totals.","turn-a-pivot-table-export-back-into-tidy-data","advanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data\u002Findex",[1719,1720],"messy-sheets","pivots-and-summaries","how-to","bhIzT8BULaAVlQnKoabnOiJmg-teKj0R1Kt-GHpJidQ",[1724,1728],{"title":1725,"path":1726,"stem":1727,"children":-1},"Stop Reading at Notes and Footers Below the Data","\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data","advanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data\u002Findex",{"title":1729,"path":1730,"stem":1731,"children":-1},"Handling Missing Data in Excel Reports with Pandas","\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports","advanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Findex",[1733,1735],{"path":1734,"title":22},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets",{"path":1736,"title":1709},"\u002Fadvanced-data-transformation-and-cleaning",[],[1739,1742,1745,1748,1751,1754],{"path":1740,"title":1741,"kind":1721},"\u002Fformatting-and-charting-excel-reports-with-python\u002Fstyling-excel-cells-with-openpyxl\u002Ffreeze-header-row-in-excel-with-openpyxl","Freeze the Header Row in Excel with openpyxl",{"path":1743,"title":1744,"kind":1721},"\u002Fgetting-started-with-python-excel-automation\u002Freading-excel-files-with-pandas\u002Fhandle-merged-cells-when-reading-excel-with-pandas","Handle Merged Cells When Reading Excel with pandas",{"path":1746,"title":1747,"kind":1721},"\u002Fgetting-started-with-python-excel-automation\u002Fusing-openpyxl-for-excel-file-manipulation\u002Funmerge-cells-and-fill-values-with-openpyxl","Unmerge Cells and Fill the Values with openpyxl",{"path":1749,"title":1750,"kind":1721},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fcombine-sheets-with-different-layouts-into-one-table","Combine Sheets with Different Layouts into One Table",{"path":1752,"title":1753,"kind":1721},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fextract-hyperlink-urls-from-excel-cells","Extract Hyperlink URLs from Excel Cells with openpyxl",{"path":1755,"title":1756,"kind":1721},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fextract-report-header-fields-from-an-excel-sheet","Extract Report Header Fields from an Excel Sheet with Python",1790972094932]