[{"data":1,"prerenderedAt":2133},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data":2099,"ancestors:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data":2108,"children:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data":2113,"related:\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data":2114},{"id":4,"title":5,"body":6,"dateModified":2066,"datePublished":2066,"description":2067,"extension":2068,"faq":2069,"keyword":2078,"kind":2079,"meta":2080,"navigation":99,"path":2088,"seo":2089,"slug":2092,"stem":2093,"topics":2094,"type":2097,"__hash__":2098},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data\u002Findex.md","Stop Reading at Notes and Footers Below the Data",{"type":7,"value":8,"toc":2054},"minimark",[9,28,33,61,67,71,175,182,186,189,627,635,903,1078,1081,1085,1092,1169,1185,1189,1192,1514,1521,1525,1532,1755,1768,1936,1940,1975,1979,1989,1993,1999,2005,2011,2017,2021,2050],[10,11,12,13,17,18,21,22,27],"p",{},"Reports prepared for people often end with more than data. Under the table there is a total row, a blank line, then a few notes: \"Source: finance system, extracted 30 Sep\", \"Figures marked * are provisional\", \"Excludes intercompany sales\". ",[14,15,16],"code",{},"pd.read_excel"," reads all of it as rows of the table. The notes land in the first column with blanks everywhere else, the total row doubles every sum, and the numeric columns turn into text because one cell says \"Source: …\". Some values carry the footnote marker themselves — ",[14,19,20],{},"1,240*"," — which makes them text too. This page finds where the table actually ends, reads only the data, keeps the notes as metadata instead of throwing them away, and turns footnote markers in cells into a clean number plus a flag. It belongs to ",[23,24,26],"a",{"href":25},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002F","Extracting Data from Messy Excel Sheets",".",[29,30,32],"h2",{"id":31},"prerequisites","Prerequisites",[34,35,40],"pre",{"className":36,"code":37,"language":38,"meta":39,"style":39},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash","",[14,41,42],{"__ignoreMap":39},[43,44,47,51,55,58],"span",{"class":45,"line":46},"line",1,[43,48,50],{"class":49},"sMTad","pip",[43,52,54],{"class":53},"srMev"," install",[43,56,57],{"class":53}," pandas",[43,59,60],{"class":53}," openpyxl\n",[10,62,63,66],{},[14,64,65],{},"branch_sales.xlsx"," holds six branches with orders and revenue, followed by a bold total row, a blank row, and four lines of notes in column A. Two revenue values carry an asterisk footnote marker and were typed as text with thousands separators, as happens when figures are copied from another document.",[29,68,70],{"id":69},"step-1-see-what-a-plain-read-does","Step 1 — See what a plain read does",[34,72,76],{"className":73,"code":74,"language":75,"meta":39,"style":39},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\nnaive = pd.read_excel(\"branch_sales.xlsx\")\nprint(naive.tail(6).to_string())\nprint(naive.dtypes.to_dict())\nprint(\"sum of Orders as read:\", pd.to_numeric(naive[\"Orders\"], errors=\"coerce\").sum())\n","python",[14,77,78,94,101,119,135,143],{"__ignoreMap":39},[43,79,80,84,88,91],{"class":45,"line":46},[43,81,83],{"class":82},"s-kum","import",[43,85,87],{"class":86},"skGVy"," pandas ",[43,89,90],{"class":82},"as",[43,92,93],{"class":86}," pd\n",[43,95,97],{"class":45,"line":96},2,[43,98,100],{"emptyLinePlaceholder":99},true,"\n",[43,102,104,107,110,113,116],{"class":45,"line":103},3,[43,105,106],{"class":86},"naive ",[43,108,109],{"class":82},"=",[43,111,112],{"class":86}," pd.read_excel(",[43,114,115],{"class":53},"\"branch_sales.xlsx\"",[43,117,118],{"class":86},")\n",[43,120,122,126,129,132],{"class":45,"line":121},4,[43,123,125],{"class":124},"sP0c6","print",[43,127,128],{"class":86},"(naive.tail(",[43,130,131],{"class":124},"6",[43,133,134],{"class":86},").to_string())\n",[43,136,138,140],{"class":45,"line":137},5,[43,139,125],{"class":124},[43,141,142],{"class":86},"(naive.dtypes.to_dict())\n",[43,144,146,148,151,154,157,160,163,167,169,172],{"class":45,"line":145},6,[43,147,125],{"class":124},[43,149,150],{"class":86},"(",[43,152,153],{"class":53},"\"sum of Orders as read:\"",[43,155,156],{"class":86},", pd.to_numeric(naive[",[43,158,159],{"class":53},"\"Orders\"",[43,161,162],{"class":86},"], ",[43,164,166],{"class":165},"sa561","errors",[43,168,109],{"class":82},[43,170,171],{"class":53},"\"coerce\"",[43,173,174],{"class":86},").sum())\n",[10,176,177,178,181],{},"The DataFrame has twelve rows instead of six: the total row, a row of NaN for the blank line and four \"rows\" of notes. Revenue is an ",[14,179,180],{},"object"," column because of the text cells, so it cannot be summed at all, and summing Orders gives 3,664 — exactly double the real 1,832, because the total row is counted as a branch. Nothing errors; every downstream figure is simply wrong. The fix is to stop reading where the table ends.",[29,183,185],{"id":184},"step-2-find-where-the-table-ends","Step 2 — Find where the table ends",[10,187,188],{},"There are three reliable signals, and it is worth combining them: the first completely blank row after the header, a row whose key column does not look like a key, and a known footer word such as \"Total\" or \"Notes\". Scan column A with openpyxl to find the first row that fails:",[34,190,192],{"className":73,"code":191,"language":75,"meta":39,"style":39},"import re\nfrom openpyxl import load_workbook\n\nKEY = re.compile(r\"^[A-Z]{3}-\\d{2}$\")          # branch codes look like LDS-01\nSTOP_WORDS = (\"total\", \"notes\", \"source\")\n\ndef data_end(path, sheet=None, header_row=1):\n    ws = load_workbook(path, read_only=True)[sheet] if sheet else load_workbook(path, read_only=True).active\n    for r, row in enumerate(ws.iter_rows(min_row=header_row + 1, values_only=True), start=header_row + 1):\n        first = row[0]\n        if all(v in (None, \"\") for v in row):\n            return r - 1, \"blank row\"\n        text = str(first).strip()\n        if text.lower().startswith(STOP_WORDS) or not KEY.match(text):\n            return r - 1, f\"stop at {text[:20]!r}\"\n    return r, \"end of sheet\"\n\nlast_row, reason = data_end(\"branch_sales.xlsx\")\nprint(\"data ends at Excel row\", last_row, \"—\", reason)\n",[14,193,194,201,214,218,269,295,299,328,370,427,444,481,499,513,537,576,588,593,608],{"__ignoreMap":39},[43,195,196,198],{"class":45,"line":46},[43,197,83],{"class":82},[43,199,200],{"class":86}," re\n",[43,202,203,206,209,211],{"class":45,"line":96},[43,204,205],{"class":82},"from",[43,207,208],{"class":86}," openpyxl ",[43,210,83],{"class":82},[43,212,213],{"class":86}," load_workbook\n",[43,215,216],{"class":45,"line":103},[43,217,100],{"emptyLinePlaceholder":99},[43,219,220,223,226,229,232,235,238,242,245,248,251,254,257,260,262,265],{"class":45,"line":121},[43,221,222],{"class":124},"KEY",[43,224,225],{"class":82}," =",[43,227,228],{"class":86}," re.compile(",[43,230,231],{"class":82},"r",[43,233,234],{"class":53},"\"",[43,236,237],{"class":124},"^[",[43,239,241],{"class":240},"sSjpA","A-Z",[43,243,244],{"class":124},"]",[43,246,247],{"class":82},"{3}",[43,249,250],{"class":53},"-",[43,252,253],{"class":124},"\\d",[43,255,256],{"class":82},"{2}",[43,258,259],{"class":124},"$",[43,261,234],{"class":53},[43,263,264],{"class":86},")          ",[43,266,268],{"class":267},"s-wDw","# branch codes look like LDS-01\n",[43,270,271,274,276,279,282,285,288,290,293],{"class":45,"line":137},[43,272,273],{"class":124},"STOP_WORDS",[43,275,225],{"class":82},[43,277,278],{"class":86}," (",[43,280,281],{"class":53},"\"total\"",[43,283,284],{"class":86},", ",[43,286,287],{"class":53},"\"notes\"",[43,289,284],{"class":86},[43,291,292],{"class":53},"\"source\"",[43,294,118],{"class":86},[43,296,297],{"class":45,"line":145},[43,298,100],{"emptyLinePlaceholder":99},[43,300,302,305,309,312,314,317,320,322,325],{"class":45,"line":301},7,[43,303,304],{"class":82},"def",[43,306,308],{"class":307},"s_Opv"," data_end",[43,310,311],{"class":86},"(path, sheet",[43,313,109],{"class":82},[43,315,316],{"class":124},"None",[43,318,319],{"class":86},", header_row",[43,321,109],{"class":82},[43,323,324],{"class":124},"1",[43,326,327],{"class":86},"):\n",[43,329,331,334,336,339,342,344,347,350,353,356,359,361,363,365,367],{"class":45,"line":330},8,[43,332,333],{"class":86},"    ws ",[43,335,109],{"class":82},[43,337,338],{"class":86}," load_workbook(path, ",[43,340,341],{"class":165},"read_only",[43,343,109],{"class":82},[43,345,346],{"class":124},"True",[43,348,349],{"class":86},")[sheet] ",[43,351,352],{"class":82},"if",[43,354,355],{"class":86}," sheet ",[43,357,358],{"class":82},"else",[43,360,338],{"class":86},[43,362,341],{"class":165},[43,364,109],{"class":82},[43,366,346],{"class":124},[43,368,369],{"class":86},").active\n",[43,371,373,376,379,382,385,388,391,393,396,399,402,404,407,409,411,414,417,419,421,423,425],{"class":45,"line":372},9,[43,374,375],{"class":82},"    for",[43,377,378],{"class":86}," r, row ",[43,380,381],{"class":82},"in",[43,383,384],{"class":124}," enumerate",[43,386,387],{"class":86},"(ws.iter_rows(",[43,389,390],{"class":165},"min_row",[43,392,109],{"class":82},[43,394,395],{"class":86},"header_row ",[43,397,398],{"class":82},"+",[43,400,401],{"class":124}," 1",[43,403,284],{"class":86},[43,405,406],{"class":165},"values_only",[43,408,109],{"class":82},[43,410,346],{"class":124},[43,412,413],{"class":86},"), ",[43,415,416],{"class":165},"start",[43,418,109],{"class":82},[43,420,395],{"class":86},[43,422,398],{"class":82},[43,424,401],{"class":124},[43,426,327],{"class":86},[43,428,430,433,435,438,441],{"class":45,"line":429},10,[43,431,432],{"class":86},"        first ",[43,434,109],{"class":82},[43,436,437],{"class":86}," row[",[43,439,440],{"class":124},"0",[43,442,443],{"class":86},"]\n",[43,445,447,450,453,456,458,460,462,464,467,470,473,476,478],{"class":45,"line":446},11,[43,448,449],{"class":82},"        if",[43,451,452],{"class":124}," all",[43,454,455],{"class":86},"(v ",[43,457,381],{"class":82},[43,459,278],{"class":86},[43,461,316],{"class":124},[43,463,284],{"class":86},[43,465,466],{"class":53},"\"\"",[43,468,469],{"class":86},") ",[43,471,472],{"class":82},"for",[43,474,475],{"class":86}," v ",[43,477,381],{"class":82},[43,479,480],{"class":86}," row):\n",[43,482,484,487,490,492,494,496],{"class":45,"line":483},12,[43,485,486],{"class":82},"            return",[43,488,489],{"class":86}," r ",[43,491,250],{"class":82},[43,493,401],{"class":124},[43,495,284],{"class":86},[43,497,498],{"class":53},"\"blank row\"\n",[43,500,502,505,507,510],{"class":45,"line":501},13,[43,503,504],{"class":86},"        text ",[43,506,109],{"class":82},[43,508,509],{"class":124}," str",[43,511,512],{"class":86},"(first).strip()\n",[43,514,516,518,521,523,525,528,531,534],{"class":45,"line":515},14,[43,517,449],{"class":82},[43,519,520],{"class":86}," text.lower().startswith(",[43,522,273],{"class":124},[43,524,469],{"class":86},[43,526,527],{"class":82},"or",[43,529,530],{"class":82}," not",[43,532,533],{"class":124}," KEY",[43,535,536],{"class":86},".match(text):\n",[43,538,540,542,544,546,548,550,553,556,559,562,565,567,570,573],{"class":45,"line":539},15,[43,541,486],{"class":82},[43,543,489],{"class":86},[43,545,250],{"class":82},[43,547,401],{"class":124},[43,549,284],{"class":86},[43,551,552],{"class":82},"f",[43,554,555],{"class":53},"\"stop at ",[43,557,558],{"class":240},"{",[43,560,561],{"class":86},"text[:",[43,563,564],{"class":124},"20",[43,566,244],{"class":86},[43,568,569],{"class":82},"!r",[43,571,572],{"class":240},"}",[43,574,575],{"class":53},"\"\n",[43,577,579,582,585],{"class":45,"line":578},16,[43,580,581],{"class":82},"    return",[43,583,584],{"class":86}," r, ",[43,586,587],{"class":53},"\"end of sheet\"\n",[43,589,591],{"class":45,"line":590},17,[43,592,100],{"emptyLinePlaceholder":99},[43,594,596,599,601,604,606],{"class":45,"line":595},18,[43,597,598],{"class":86},"last_row, reason ",[43,600,109],{"class":82},[43,602,603],{"class":86}," data_end(",[43,605,115],{"class":53},[43,607,118],{"class":86},[43,609,611,613,615,618,621,624],{"class":45,"line":610},19,[43,612,125],{"class":124},[43,614,150],{"class":86},[43,616,617],{"class":53},"\"data ends at Excel row\"",[43,619,620],{"class":86},", last_row, ",[43,622,623],{"class":53},"\"—\"",[43,625,626],{"class":86},", reason)\n",[10,628,629,630,634],{},"The scan stops at row 8, the total row, because \"Total\" is neither a branch code nor allowed through the stop words; the last data row is row 7. Checking the key's ",[631,632,633],"em",{},"shape"," with a regular expression is the strongest of the three signals: it also catches a footer the stop-word list did not anticipate, such as \"Excludes intercompany sales\", and a total row with no blank line before it. Where keys have no fixed format, fall back on the blank row and the stop words. Reading in read-only mode keeps the scan fast even for large sheets.",[636,637,645,646,645,650,645,654,645,661,645,670,645,674,645,681,645,685,645,689,645,693,645,697,645,701,645,705,645,709,645,713,645,717,645,721,645,726,645,730,645,732,645,735,645,737,645,740,645,742,645,745,645,747,645,750,645,753,645,757,645,759,645,762,645,764,645,767,645,769,645,772,645,774,645,777,645,779,645,782,645,785,645,789,645,791,645,794,645,796,645,799,645,801,645,804,645,806,645,809,645,813,645,817,645,820,645,824,645,826,645,829,645,831,645,834,645,836,645,839,645,841,645,844,645,848,645,852,645,855,645,859,645,861,645,864,645,866,645,868,645,870,645,872,645,875,645,878,645,881,645,883,645,886,645,888,645,891,645,893,645,895,645,899],"svg",{"viewBox":638,"role":639,"ariaLabelledBy":640,"xmlns":643,"style":644},"142 0 475 224","img",[641,642],"ft-end-t","ft-end-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:475px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[647,648,649],"title",{"id":641},"Where the data ends and the footer begins",[651,652,653],"desc",{"id":642},"Rows 2 to 7 hold branch codes such as LDS-01 and are data. Row 8 is a bold total row, row 9 is blank, and rows 10 to 13 are notes. The scan stops at row 8 because Total does not match the branch code pattern.",[655,656],"rect",{"x":657,"y":440,"width":658,"height":659,"fill":660},"142","475","224","#ffffff",[655,662],{"x":663,"y":664,"width":665,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"158.5","16.0","32.0","24.0","#e7ebef","var(--line,#cdd5e6)","stroke-width:1px",[655,671],{"x":672,"y":664,"width":673,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"190.5","53.6",[675,676,680],"text",{"x":677,"y":678,"style":679},"217.3","33.0","font-size:11px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","A",[655,682],{"x":683,"y":664,"width":684,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"244.1","131.6",[675,686,688],{"x":687,"y":678,"style":679},"309.9","B",[655,690],{"x":691,"y":664,"width":692,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"375.7","61.2",[675,694,696],{"x":695,"y":678,"style":679},"406.3","C",[655,698],{"x":699,"y":664,"width":700,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"436.9","71.3",[675,702,704],{"x":703,"y":678,"style":679},"472.6","D",[655,706],{"x":707,"y":664,"width":708,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"508.2","93.2",[675,710,712],{"x":711,"y":678,"style":679},"554.8","E",[655,714],{"x":663,"y":715,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"40.0","28.0",[675,718,324],{"x":719,"y":720,"style":679},"174.5","59.0",[655,722],{"x":672,"y":715,"width":673,"height":716,"rx":440,"fill":723,"stroke":724,"style":725},"#f0f4ff","var(--brand,#5b5cf0)","stroke-width:1.5px",[675,727,729],{"x":677,"y":720,"style":728},"font-size:10.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Row",[655,731],{"x":683,"y":715,"width":684,"height":716,"rx":440,"fill":723,"stroke":724,"style":725},[675,733,734],{"x":687,"y":720,"style":728},"Branch",[655,736],{"x":691,"y":715,"width":692,"height":716,"rx":440,"fill":723,"stroke":724,"style":725},[675,738,739],{"x":695,"y":720,"style":728},"Orders",[655,741],{"x":699,"y":715,"width":700,"height":716,"rx":440,"fill":723,"stroke":724,"style":725},[675,743,744],{"x":703,"y":720,"style":728},"Revenue",[655,746],{"x":707,"y":715,"width":708,"height":716,"rx":440,"fill":723,"stroke":724,"style":725},[675,748,749],{"x":711,"y":720,"style":728},"Role",[655,751],{"x":663,"y":752,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"68.0",[675,754,756],{"x":719,"y":755,"style":679},"87.0","2",[655,758],{"x":672,"y":752,"width":673,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,760,756],{"x":677,"y":755,"style":761},"font-size:10.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle",[655,763],{"x":683,"y":752,"width":684,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,765,766],{"x":687,"y":755,"style":761},"LDS-01",[655,768],{"x":691,"y":752,"width":692,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,770,771],{"x":695,"y":755,"style":761},"412",[655,773],{"x":699,"y":752,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,775,776],{"x":703,"y":755,"style":761},"51,840",[655,778],{"x":707,"y":752,"width":708,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,780,781],{"x":711,"y":755,"style":761},"data",[655,783],{"x":663,"y":784,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"96.0",[675,786,788],{"x":719,"y":787,"style":679},"115.0","3",[655,790],{"x":672,"y":784,"width":673,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,792,793],{"x":677,"y":787,"style":761},"7",[655,795],{"x":683,"y":784,"width":684,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,797,798],{"x":687,"y":787,"style":761},"CDF-01",[655,800],{"x":691,"y":784,"width":692,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,802,803],{"x":695,"y":787,"style":761},"189",[655,805],{"x":699,"y":784,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,807,808],{"x":703,"y":787,"style":761},"22,760",[655,810],{"x":707,"y":784,"width":708,"height":716,"rx":440,"fill":811,"stroke":812,"style":725},"#d9f4f1","var(--teal,#0f9488)",[675,814,816],{"x":711,"y":787,"style":815},"font-size:10.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","last data row",[655,818],{"x":663,"y":819,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"124.0",[675,821,823],{"x":719,"y":822,"style":679},"143.0","4",[655,825],{"x":672,"y":819,"width":673,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,827,828],{"x":677,"y":822,"style":761},"8",[655,830],{"x":683,"y":819,"width":684,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,832,833],{"x":687,"y":822,"style":761},"Total",[655,835],{"x":691,"y":819,"width":692,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,837,838],{"x":695,"y":822,"style":761},"1,832",[655,840],{"x":699,"y":819,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,842,843],{"x":703,"y":822,"style":761},"232,325",[655,845],{"x":707,"y":819,"width":708,"height":716,"rx":440,"fill":846,"stroke":847,"style":725},"#fce9e9","var(--danger,#dc2626)",[675,849,851],{"x":711,"y":822,"style":850},"font-size:10.5px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","stop",[655,853],{"x":663,"y":854,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"152.0",[675,856,858],{"x":719,"y":857,"style":679},"171.0","5",[655,860],{"x":672,"y":854,"width":673,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,862,863],{"x":677,"y":857,"style":761},"9",[655,865],{"x":683,"y":854,"width":684,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[655,867],{"x":691,"y":854,"width":692,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[655,869],{"x":699,"y":854,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[655,871],{"x":707,"y":854,"width":708,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,873,874],{"x":711,"y":857,"style":761},"blank",[655,876],{"x":663,"y":877,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"180.0",[675,879,131],{"x":719,"y":880,"style":679},"199.0",[655,882],{"x":672,"y":877,"width":673,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,884,885],{"x":677,"y":880,"style":761},"10–13",[655,887],{"x":683,"y":877,"width":684,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,889,890],{"x":687,"y":880,"style":761},"Notes: … Source: …",[655,892],{"x":691,"y":877,"width":692,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[655,894],{"x":699,"y":877,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[655,896],{"x":707,"y":877,"width":708,"height":716,"rx":440,"fill":897,"stroke":898,"style":725},"#fdefd8","var(--gold,#b4740a)",[675,900,902],{"x":711,"y":880,"style":901},"font-size:10.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","footer",[636,904,645,910,645,913,645,916,645,920,645,927,645,938,645,943,645,947,645,951,645,954,645,959,645,967,645,972,645,978,645,985,645,990,645,995,645,998,645,1001,645,1005,645,1008,645,1011,645,1018,645,1021,645,1023,645,1026,645,1029,645,1033,645,1036,645,1039,645,1044,645,1047,645,1049,645,1051,645,1054,645,1058,645,1061,645,1064,645,1071,645,1075],{"viewBox":905,"role":639,"ariaLabelledBy":906,"xmlns":643,"style":909},"4 0 752 272",[907,908],"ft-signal-t","ft-signal-d","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[647,911,912],{"id":907},"Is this row still data?",[651,914,915],{"id":908},"For each row after the header, a completely blank row ends the table. A first cell starting with Total, Notes or Source ends it. A key that does not match the expected pattern, such as LDS-01, ends it. Otherwise the row is data and the scan continues.",[655,917],{"x":823,"y":440,"width":918,"height":919,"fill":660},"752","272",[655,921],{"x":922,"y":664,"width":923,"height":924,"rx":925,"fill":723,"stroke":724,"style":926},"240.0","280.0","75.5","12","stroke-width:2px",[928,929,931],"g",{"transform":930},"translate(368.0,29.0) scale(1.000)",[932,933],"path",{"d":934,"fill":935,"stroke":936,"style":937},"M11 4a7 7 0 1 0 .01 0z M16 16l5 5","none","var(--brand-strong,#4338ca)","stroke-width:1.80px;stroke-linecap:round;stroke-linejoin:round",[675,939,942],{"x":940,"y":924,"style":941},"380.0","font-size:13.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Check the row after the header",[45,944],{"x1":940,"y1":945,"x2":940,"y2":946,"stroke":724,"style":926},"91.5","111.5",[45,948],{"x1":949,"y1":946,"x2":950,"y2":946,"stroke":724,"style":926},"104.0","656.0",[45,952],{"x1":949,"y1":946,"x2":949,"y2":953,"stroke":724,"style":926},"157.5",[955,956],"polygon",{"points":957,"fill":958},"104.0,161.5 99.0,152.5 109.0,152.5","#5b5cf0",[655,960],{"x":961,"y":962,"width":963,"height":964,"rx":965,"fill":660,"stroke":668,"style":966},"55.3","121.5","97.3","21.0","10","stroke-width:1.2px",[675,968,971],{"x":949,"y":969,"style":970},"136.0","font-size:11.5px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","all cells blank",[655,973],{"x":974,"y":975,"width":976,"height":977,"rx":925,"fill":897,"stroke":898,"style":926},"20.0","163.5","168.0","93.0",[928,979,981],{"transform":980},"translate(92.0,176.5) scale(1.000)",[932,982],{"d":983,"fill":935,"stroke":984,"style":937},"M6 6l12 12 M18 6L6 18","var(--gold-ink,#7a4e06)",[675,986,989],{"x":949,"y":987,"style":988},"223.0","font-size:13.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","end of data",[675,991,994],{"x":949,"y":992,"style":993},"242.5","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","stop before this row",[45,996],{"x1":997,"y1":946,"x2":997,"y2":953,"stroke":724,"style":926},"288.0",[955,999],{"points":1000,"fill":958},"288.0,161.5 283.0,152.5 293.0,152.5",[655,1002],{"x":1003,"y":962,"width":1004,"height":964,"rx":965,"fill":660,"stroke":668,"style":966},"215.0","146.0",[675,1006,1007],{"x":997,"y":969,"style":970},"Total \u002F Notes \u002F Source",[655,1009],{"x":1010,"y":975,"width":976,"height":977,"rx":925,"fill":846,"stroke":847,"style":926},"204.0",[928,1012,1014],{"transform":1013},"translate(276.0,176.5) scale(1.000)",[932,1015],{"d":1016,"fill":935,"stroke":1017,"style":937},"M12 3.5l9.5 17h-19z M12 10v5 M12 17.8v.4","var(--accent-ink,#be185d)",[675,1019,902],{"x":997,"y":987,"style":1020},"font-size:13.5px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle",[675,1022,994],{"x":997,"y":992,"style":993},[45,1024],{"x1":1025,"y1":946,"x2":1025,"y2":953,"stroke":724,"style":926},"472.0",[955,1027],{"points":1028,"fill":958},"472.0,161.5 467.0,152.5 477.0,152.5",[655,1030],{"x":1031,"y":962,"width":1032,"height":964,"rx":965,"fill":660,"stroke":668,"style":966},"404.3","135.4",[675,1034,1035],{"x":1025,"y":969,"style":970},"key fails the pattern",[655,1037],{"x":1038,"y":975,"width":976,"height":977,"rx":925,"fill":846,"stroke":847,"style":926},"388.0",[928,1040,1042],{"transform":1041},"translate(460.0,176.5) scale(1.000)",[932,1043],{"d":1016,"fill":935,"stroke":1017,"style":937},[675,1045,1046],{"x":1025,"y":987,"style":1020},"footer or junk",[675,1048,994],{"x":1025,"y":992,"style":993},[45,1050],{"x1":950,"y1":946,"x2":950,"y2":953,"stroke":724,"style":926},[955,1052],{"points":1053,"fill":958},"656.0,161.5 651.0,152.5 661.0,152.5",[655,1055],{"x":1056,"y":962,"width":1057,"height":964,"rx":965,"fill":660,"stroke":668,"style":966},"603.6","104.8",[675,1059,1060],{"x":950,"y":969,"style":970},"key looks right",[655,1062],{"x":1063,"y":975,"width":976,"height":977,"rx":925,"fill":811,"stroke":812,"style":926},"572.0",[928,1065,1067],{"transform":1066},"translate(644.0,176.5) scale(1.000)",[932,1068],{"d":1069,"fill":935,"stroke":1070,"style":937},"M5 12.5l4.5 4.5L19 7","var(--teal-ink,#0b6157)",[675,1072,1074],{"x":950,"y":987,"style":1073},"font-size:13.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","data row",[675,1076,1077],{"x":950,"y":992,"style":993},"keep scanning",[10,1079,1080],{},"The order of the checks does not matter much in practice, but having all three makes the scan robust to the variations reports go through over time: a footer added without a blank line, a blank line inserted before the total, a renamed \"Grand total\". When the scan stops somewhere unexpected — after one row, or at the very end of a long sheet — log the reason it gives; that message is usually enough to see what changed in the report layout.",[29,1082,1084],{"id":1083},"step-3-read-exactly-the-data-rows","Step 3 — Read exactly the data rows",[10,1086,1087,1088,1091],{},"With the last row known, ",[14,1089,1090],{},"nrows"," tells pandas how many data rows to read after the header:",[34,1093,1095],{"className":73,"code":1094,"language":75,"meta":39,"style":39},"df = pd.read_excel(\"branch_sales.xlsx\", nrows=last_row - 1)\nprint(df.to_string(index=False))\nprint(\"rows:\", len(df), \"| Orders total:\", df[\"Orders\"].sum())\n",[14,1096,1097,1123,1141],{"__ignoreMap":39},[43,1098,1099,1102,1104,1106,1108,1110,1112,1114,1117,1119,1121],{"class":45,"line":46},[43,1100,1101],{"class":86},"df ",[43,1103,109],{"class":82},[43,1105,112],{"class":86},[43,1107,115],{"class":53},[43,1109,284],{"class":86},[43,1111,1090],{"class":165},[43,1113,109],{"class":82},[43,1115,1116],{"class":86},"last_row ",[43,1118,250],{"class":82},[43,1120,401],{"class":124},[43,1122,118],{"class":86},[43,1124,1125,1127,1130,1133,1135,1138],{"class":45,"line":96},[43,1126,125],{"class":124},[43,1128,1129],{"class":86},"(df.to_string(",[43,1131,1132],{"class":165},"index",[43,1134,109],{"class":82},[43,1136,1137],{"class":124},"False",[43,1139,1140],{"class":86},"))\n",[43,1142,1143,1145,1147,1150,1152,1155,1158,1161,1164,1166],{"class":45,"line":103},[43,1144,125],{"class":124},[43,1146,150],{"class":86},[43,1148,1149],{"class":53},"\"rows:\"",[43,1151,284],{"class":86},[43,1153,1154],{"class":124},"len",[43,1156,1157],{"class":86},"(df), ",[43,1159,1160],{"class":53},"\"| Orders total:\"",[43,1162,1163],{"class":86},", df[",[43,1165,159],{"class":53},[43,1167,1168],{"class":86},"].sum())\n",[10,1170,1171,1173,1174,1177,1178,1180,1181,27],{},[14,1172,1090],{}," counts data rows, excluding the header, so the header on row 1 and data ending on row 7 give six rows. Orders now sums to 1,832, matching the report's own total row — a useful check in itself. Revenue is still text because of the two marked values; Step 5 handles that. If the header is not on the first row, combine ",[14,1175,1176],{},"skiprows"," and ",[14,1179,1090],{},", as described in ",[23,1182,1184],{"href":1183},"\u002Fgetting-started-with-python-excel-automation\u002Freading-excel-files-with-pandas\u002Fskip-rows-and-set-header-when-reading-excel-with-pandas\u002F","Skip Rows and Set the Header When Reading Excel with pandas",[29,1186,1188],{"id":1187},"step-4-keep-the-footer-as-metadata","Step 4 — Keep the footer as metadata",[10,1190,1191],{},"The notes matter — they say where the figures came from and which ones are provisional — so capture them rather than discard them. Read everything below the data that has text in column A:",[34,1193,1195],{"className":73,"code":1194,"language":75,"meta":39,"style":39},"wb = load_workbook(\"branch_sales.xlsx\", read_only=True)\nbelow = [row[0] for row in wb.active.iter_rows(min_row=last_row + 1, values_only=True) if row and row[0]]\nwb.close()\ntotal_row = below[0] if below and str(below[0]).lower().startswith(\"total\") else None\nnotes = [str(n) for n in below if str(n).lower() not in (\"total\", \"notes:\")]\nmeta = {\n    \"source\": next((n.split(\":\", 1)[1].strip() for n in notes if n.lower().startswith(\"source\")), None),\n    \"provisional_note\": next((n for n in notes if n.startswith(\"*\")), None),\n    \"other_notes\": [n for n in notes if not n.lower().startswith((\"source\", \"*\"))],\n}\nprint(meta)\n",[14,1196,1197,1219,1278,1283,1323,1373,1383,1436,1470,1502,1507],{"__ignoreMap":39},[43,1198,1199,1202,1204,1207,1209,1211,1213,1215,1217],{"class":45,"line":46},[43,1200,1201],{"class":86},"wb ",[43,1203,109],{"class":82},[43,1205,1206],{"class":86}," load_workbook(",[43,1208,115],{"class":53},[43,1210,284],{"class":86},[43,1212,341],{"class":165},[43,1214,109],{"class":82},[43,1216,346],{"class":124},[43,1218,118],{"class":86},[43,1220,1221,1224,1226,1229,1231,1234,1236,1239,1241,1244,1246,1248,1250,1252,1254,1256,1258,1260,1262,1264,1266,1268,1271,1273,1275],{"class":45,"line":96},[43,1222,1223],{"class":86},"below ",[43,1225,109],{"class":82},[43,1227,1228],{"class":86}," [row[",[43,1230,440],{"class":124},[43,1232,1233],{"class":86},"] ",[43,1235,472],{"class":82},[43,1237,1238],{"class":86}," row ",[43,1240,381],{"class":82},[43,1242,1243],{"class":86}," wb.active.iter_rows(",[43,1245,390],{"class":165},[43,1247,109],{"class":82},[43,1249,1116],{"class":86},[43,1251,398],{"class":82},[43,1253,401],{"class":124},[43,1255,284],{"class":86},[43,1257,406],{"class":165},[43,1259,109],{"class":82},[43,1261,346],{"class":124},[43,1263,469],{"class":86},[43,1265,352],{"class":82},[43,1267,1238],{"class":86},[43,1269,1270],{"class":82},"and",[43,1272,437],{"class":86},[43,1274,440],{"class":124},[43,1276,1277],{"class":86},"]]\n",[43,1279,1280],{"class":45,"line":103},[43,1281,1282],{"class":86},"wb.close()\n",[43,1284,1285,1288,1290,1293,1295,1297,1299,1302,1304,1306,1309,1311,1314,1316,1318,1320],{"class":45,"line":121},[43,1286,1287],{"class":86},"total_row ",[43,1289,109],{"class":82},[43,1291,1292],{"class":86}," below[",[43,1294,440],{"class":124},[43,1296,1233],{"class":86},[43,1298,352],{"class":82},[43,1300,1301],{"class":86}," below ",[43,1303,1270],{"class":82},[43,1305,509],{"class":124},[43,1307,1308],{"class":86},"(below[",[43,1310,440],{"class":124},[43,1312,1313],{"class":86},"]).lower().startswith(",[43,1315,281],{"class":53},[43,1317,469],{"class":86},[43,1319,358],{"class":82},[43,1321,1322],{"class":124}," None\n",[43,1324,1325,1328,1330,1333,1336,1339,1341,1344,1346,1348,1350,1352,1355,1358,1361,1363,1365,1367,1370],{"class":45,"line":137},[43,1326,1327],{"class":86},"notes ",[43,1329,109],{"class":82},[43,1331,1332],{"class":86}," [",[43,1334,1335],{"class":124},"str",[43,1337,1338],{"class":86},"(n) ",[43,1340,472],{"class":82},[43,1342,1343],{"class":86}," n ",[43,1345,381],{"class":82},[43,1347,1301],{"class":86},[43,1349,352],{"class":82},[43,1351,509],{"class":124},[43,1353,1354],{"class":86},"(n).lower() ",[43,1356,1357],{"class":82},"not",[43,1359,1360],{"class":82}," in",[43,1362,278],{"class":86},[43,1364,281],{"class":53},[43,1366,284],{"class":86},[43,1368,1369],{"class":53},"\"notes:\"",[43,1371,1372],{"class":86},")]\n",[43,1374,1375,1378,1380],{"class":45,"line":145},[43,1376,1377],{"class":86},"meta ",[43,1379,109],{"class":82},[43,1381,1382],{"class":86}," {\n",[43,1384,1385,1388,1391,1394,1397,1400,1402,1404,1407,1409,1412,1414,1416,1418,1421,1423,1426,1428,1431,1433],{"class":45,"line":301},[43,1386,1387],{"class":53},"    \"source\"",[43,1389,1390],{"class":86},": ",[43,1392,1393],{"class":124},"next",[43,1395,1396],{"class":86},"((n.split(",[43,1398,1399],{"class":53},"\":\"",[43,1401,284],{"class":86},[43,1403,324],{"class":124},[43,1405,1406],{"class":86},")[",[43,1408,324],{"class":124},[43,1410,1411],{"class":86},"].strip() ",[43,1413,472],{"class":82},[43,1415,1343],{"class":86},[43,1417,381],{"class":82},[43,1419,1420],{"class":86}," notes ",[43,1422,352],{"class":82},[43,1424,1425],{"class":86}," n.lower().startswith(",[43,1427,292],{"class":53},[43,1429,1430],{"class":86},")), ",[43,1432,316],{"class":124},[43,1434,1435],{"class":86},"),\n",[43,1437,1438,1441,1443,1445,1448,1450,1452,1454,1456,1458,1461,1464,1466,1468],{"class":45,"line":330},[43,1439,1440],{"class":53},"    \"provisional_note\"",[43,1442,1390],{"class":86},[43,1444,1393],{"class":124},[43,1446,1447],{"class":86},"((n ",[43,1449,472],{"class":82},[43,1451,1343],{"class":86},[43,1453,381],{"class":82},[43,1455,1420],{"class":86},[43,1457,352],{"class":82},[43,1459,1460],{"class":86}," n.startswith(",[43,1462,1463],{"class":53},"\"*\"",[43,1465,1430],{"class":86},[43,1467,316],{"class":124},[43,1469,1435],{"class":86},[43,1471,1472,1475,1478,1480,1482,1484,1486,1488,1490,1493,1495,1497,1499],{"class":45,"line":372},[43,1473,1474],{"class":53},"    \"other_notes\"",[43,1476,1477],{"class":86},": [n ",[43,1479,472],{"class":82},[43,1481,1343],{"class":86},[43,1483,381],{"class":82},[43,1485,1420],{"class":86},[43,1487,352],{"class":82},[43,1489,530],{"class":82},[43,1491,1492],{"class":86}," n.lower().startswith((",[43,1494,292],{"class":53},[43,1496,284],{"class":86},[43,1498,1463],{"class":53},[43,1500,1501],{"class":86},"))],\n",[43,1503,1504],{"class":45,"line":429},[43,1505,1506],{"class":86},"}\n",[43,1508,1509,1511],{"class":45,"line":446},[43,1510,125],{"class":124},[43,1512,1513],{"class":86},"(meta)\n",[10,1515,1516,1517,27],{},"The source line, the meaning of the asterisk and the remaining caveat end up in a small dictionary that can travel with the data — written to a \"Notes\" sheet in the output, logged with the run, or attached to a database load. A report built from this data should carry the same caveats; losing \"Excludes intercompany sales\" on the way through a pipeline is how wrong conclusions get made. Header fields above a table are handled the same way in ",[23,1518,1520],{"href":1519},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fextract-report-header-fields-from-an-excel-sheet\u002F","Extract Report Header Fields from an Excel Sheet",[29,1522,1524],{"id":1523},"step-5-turn-footnote-markers-into-a-flag","Step 5 — Turn footnote markers into a flag",[10,1526,1527,1528,1531],{},"Values like ",[14,1529,1530],{},"38,920*"," mean \"38,920, provisional\". Split each into a clean number and a boolean flag:",[34,1533,1535],{"className":73,"code":1534,"language":75,"meta":39,"style":39},"def split_marker(value, marker=\"*\"):\n    if isinstance(value, (int, float)):\n        return float(value), False\n    text = str(value).strip()\n    flagged = text.endswith(marker)\n    number = float(text.rstrip(marker).replace(\",\", \"\").strip())\n    return number, flagged\n\ndf[[\"Revenue\", \"Revenue provisional\"]] = df[\"Revenue\"].apply(lambda v: pd.Series(split_marker(v)))\nprint(df.to_string(index=False))\nprint(\"revenue total:\", df[\"Revenue\"].sum(), \"| provisional share:\",\n      round(df.loc[df[\"Revenue provisional\"], \"Revenue\"].sum() \u002F df[\"Revenue\"].sum() * 100, 1), \"%\")\n",[14,1536,1537,1553,1575,1589,1601,1611,1633,1640,1644,1676,1690,1712],{"__ignoreMap":39},[43,1538,1539,1541,1544,1547,1549,1551],{"class":45,"line":46},[43,1540,304],{"class":82},[43,1542,1543],{"class":307}," split_marker",[43,1545,1546],{"class":86},"(value, marker",[43,1548,109],{"class":82},[43,1550,1463],{"class":53},[43,1552,327],{"class":86},[43,1554,1555,1558,1561,1564,1567,1569,1572],{"class":45,"line":96},[43,1556,1557],{"class":82},"    if",[43,1559,1560],{"class":124}," isinstance",[43,1562,1563],{"class":86},"(value, (",[43,1565,1566],{"class":124},"int",[43,1568,284],{"class":86},[43,1570,1571],{"class":124},"float",[43,1573,1574],{"class":86},")):\n",[43,1576,1577,1580,1583,1586],{"class":45,"line":103},[43,1578,1579],{"class":82},"        return",[43,1581,1582],{"class":124}," float",[43,1584,1585],{"class":86},"(value), ",[43,1587,1588],{"class":124},"False\n",[43,1590,1591,1594,1596,1598],{"class":45,"line":121},[43,1592,1593],{"class":86},"    text ",[43,1595,109],{"class":82},[43,1597,509],{"class":124},[43,1599,1600],{"class":86},"(value).strip()\n",[43,1602,1603,1606,1608],{"class":45,"line":137},[43,1604,1605],{"class":86},"    flagged ",[43,1607,109],{"class":82},[43,1609,1610],{"class":86}," text.endswith(marker)\n",[43,1612,1613,1616,1618,1620,1623,1626,1628,1630],{"class":45,"line":145},[43,1614,1615],{"class":86},"    number ",[43,1617,109],{"class":82},[43,1619,1582],{"class":124},[43,1621,1622],{"class":86},"(text.rstrip(marker).replace(",[43,1624,1625],{"class":53},"\",\"",[43,1627,284],{"class":86},[43,1629,466],{"class":53},[43,1631,1632],{"class":86},").strip())\n",[43,1634,1635,1637],{"class":45,"line":301},[43,1636,581],{"class":82},[43,1638,1639],{"class":86}," number, flagged\n",[43,1641,1642],{"class":45,"line":330},[43,1643,100],{"emptyLinePlaceholder":99},[43,1645,1646,1649,1652,1654,1657,1660,1662,1665,1667,1670,1673],{"class":45,"line":372},[43,1647,1648],{"class":86},"df[[",[43,1650,1651],{"class":53},"\"Revenue\"",[43,1653,284],{"class":86},[43,1655,1656],{"class":53},"\"Revenue provisional\"",[43,1658,1659],{"class":86},"]] ",[43,1661,109],{"class":82},[43,1663,1664],{"class":86}," df[",[43,1666,1651],{"class":53},[43,1668,1669],{"class":86},"].apply(",[43,1671,1672],{"class":82},"lambda",[43,1674,1675],{"class":86}," v: pd.Series(split_marker(v)))\n",[43,1677,1678,1680,1682,1684,1686,1688],{"class":45,"line":429},[43,1679,125],{"class":124},[43,1681,1129],{"class":86},[43,1683,1132],{"class":165},[43,1685,109],{"class":82},[43,1687,1137],{"class":124},[43,1689,1140],{"class":86},[43,1691,1692,1694,1696,1699,1701,1703,1706,1709],{"class":45,"line":446},[43,1693,125],{"class":124},[43,1695,150],{"class":86},[43,1697,1698],{"class":53},"\"revenue total:\"",[43,1700,1163],{"class":86},[43,1702,1651],{"class":53},[43,1704,1705],{"class":86},"].sum(), ",[43,1707,1708],{"class":53},"\"| provisional share:\"",[43,1710,1711],{"class":86},",\n",[43,1713,1714,1717,1720,1722,1724,1726,1729,1732,1734,1736,1738,1741,1744,1746,1748,1750,1753],{"class":45,"line":483},[43,1715,1716],{"class":124},"      round",[43,1718,1719],{"class":86},"(df.loc[df[",[43,1721,1656],{"class":53},[43,1723,162],{"class":86},[43,1725,1651],{"class":53},[43,1727,1728],{"class":86},"].sum() ",[43,1730,1731],{"class":82},"\u002F",[43,1733,1664],{"class":86},[43,1735,1651],{"class":53},[43,1737,1728],{"class":86},[43,1739,1740],{"class":82},"*",[43,1742,1743],{"class":124}," 100",[43,1745,284],{"class":86},[43,1747,324],{"class":124},[43,1749,413],{"class":86},[43,1751,1752],{"class":53},"\"%\"",[43,1754,118],{"class":86},[10,1756,1757,1758,1761,1762,1765,1766,27],{},"Revenue becomes a numeric column again and a new boolean column records which values were provisional. The total, 232,325, now matches the report's total row exactly — so the data, the cleaned values and the footer agree. The provisional share — about 30% of revenue here — is the kind of figure a reader of any summary built on this data deserves to know. The same pattern handles other markers: ",[14,1759,1760],{},"(e)"," for estimates, ",[14,1763,1764],{},"†"," for restated figures, or a trailing ",[14,1767,10],{},[636,1769,645,1775,645,1778,645,1781,645,1786,645,1789,645,1793,645,1797,645,1800,645,1803,645,1806,645,1810,645,1812,645,1815,645,1817,645,1819,645,1822,645,1825,645,1827,645,1829,645,1831,645,1834,645,1837,645,1840,645,1842,645,1845,645,1847,645,1849,645,1852,645,1855,645,1857,645,1860,645,1862,645,1865,645,1868,645,1871,645,1875,645,1879,645,1881,645,1884,645,1886,645,1888,645,1890,645,1893,645,1895,645,1897,645,1899,645,1902,645,1904,645,1906,645,1912,645,1916,645,1920,645,1923,645,1930],{"viewBox":1770,"role":639,"ariaLabelledBy":1771,"xmlns":643,"style":1774},"106 0 548 192",[1772,1773],"ft-marker-t","ft-marker-d","width:100%;max-width:548px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[647,1776,1777],{"id":1772},"Footnote markers become a flag column",[651,1779,1780],{"id":1773},"Revenue cells such as 38,920 with an asterisk are text. Splitting the marker off gives the number 38920 and a provisional flag set to true; plain numeric cells keep their value and get the flag false.",[655,1782],{"x":1783,"y":440,"width":1784,"height":1785,"fill":660},"106","548","192",[655,1787],{"x":1788,"y":715,"width":665,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"122.5",[655,1790],{"x":1791,"y":715,"width":1792,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"154.5","64.1",[675,1794,680],{"x":1795,"y":1796,"style":679},"186.5","57.0",[655,1798],{"x":1799,"y":715,"width":700,"height":666,"rx":440,"fill":667,"stroke":668,"style":669},"218.5",[675,1801,688],{"x":1802,"y":1796,"style":679},"254.2",[655,1804],{"x":1788,"y":1805,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"64.0",[675,1807,324],{"x":1808,"y":1809,"style":679},"138.5","83.0",[655,1811],{"x":1791,"y":1805,"width":1792,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1813,734],{"x":1795,"y":1809,"style":1814},"font-size:10.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle",[655,1816],{"x":1799,"y":1805,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1818,744],{"x":1802,"y":1809,"style":1814},[655,1820],{"x":1788,"y":1821,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"92.0",[675,1823,756],{"x":1808,"y":1824,"style":679},"111.0",[655,1826],{"x":1791,"y":1821,"width":1792,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1828,766],{"x":1795,"y":1824,"style":761},[655,1830],{"x":1799,"y":1821,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1832,1833],{"x":1802,"y":1824,"style":761},"51840",[655,1835],{"x":1788,"y":1836,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"120.0",[675,1838,788],{"x":1808,"y":1839,"style":679},"139.0",[655,1841],{"x":1791,"y":1836,"width":1792,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1843,1844],{"x":1795,"y":1839,"style":761},"BRS-01",[655,1846],{"x":1799,"y":1836,"width":700,"height":716,"rx":440,"fill":897,"stroke":898,"style":725},[675,1848,1530],{"x":1802,"y":1839,"style":901},[655,1850],{"x":1788,"y":1851,"width":665,"height":716,"rx":440,"fill":667,"stroke":668,"style":669},"148.0",[675,1853,823],{"x":1808,"y":1854,"style":679},"167.0",[655,1856],{"x":1791,"y":1851,"width":1792,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1858,1859],{"x":1795,"y":1854,"style":761},"GLA-01",[655,1861],{"x":1799,"y":1851,"width":700,"height":716,"rx":440,"fill":897,"stroke":898,"style":725},[675,1863,1864],{"x":1802,"y":1854,"style":901},"31,480*",[655,1866],{"x":1867,"y":715,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},"482.1",[675,1869,744],{"x":1870,"y":720,"style":1814},"517.7",[655,1872],{"x":1873,"y":715,"width":1874,"height":716,"rx":440,"fill":667,"stroke":668,"style":725},"553.4","84.2",[675,1876,1878],{"x":1877,"y":720,"style":1814},"595.5","Provisional",[655,1880],{"x":1867,"y":752,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1882,1883],{"x":1870,"y":755,"style":761},"51840.0",[655,1885],{"x":1873,"y":752,"width":1874,"height":716,"rx":440,"fill":811,"stroke":812,"style":725},[675,1887,1137],{"x":1877,"y":755,"style":815},[655,1889],{"x":1867,"y":784,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1891,1892],{"x":1870,"y":787,"style":761},"38920.0",[655,1894],{"x":1873,"y":784,"width":1874,"height":716,"rx":440,"fill":811,"stroke":812,"style":725},[675,1896,346],{"x":1877,"y":787,"style":815},[655,1898],{"x":1867,"y":819,"width":700,"height":716,"rx":440,"fill":660,"stroke":668,"style":669},[675,1900,1901],{"x":1870,"y":822,"style":761},"31480.0",[655,1903],{"x":1873,"y":819,"width":1874,"height":716,"rx":440,"fill":811,"stroke":812,"style":725},[675,1905,346],{"x":1877,"y":822,"style":815},[675,1907,1911],{"x":1908,"y":1909,"style":1910},"206.2","29.0","font-size:12.5px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","as typed",[675,1913,1915],{"x":1914,"y":1909,"style":1910},"559.8","cleaned",[45,1917],{"x1":1918,"y1":784,"x2":1919,"y2":784,"stroke":724,"style":926},"299.8","468.1",[955,1921],{"points":1922,"fill":958},"472.1,96.0 463.1,101.0 463.1,91.0",[655,1924],{"x":1925,"y":1926,"width":1927,"height":1928,"rx":1929,"fill":660,"stroke":668,"style":966},"325.1","63.0","121.8","22.0","11",[675,1931,1935],{"x":1932,"y":1933,"style":1934},"386.0","78.0","font-size:11.5px;font-weight:400;fill:var(--brand-strong,#4338ca);text-anchor:middle;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace","split_marker()",[29,1937,1939],{"id":1938},"common-pitfalls","Common pitfalls",[1941,1942,1943,1951,1957,1963,1969],"ul",{},[1944,1945,1946,1950],"li",{},[1947,1948,1949],"strong",{},"Reading to the end of the sheet."," Totals and notes become rows; find the real end first.",[1944,1952,1953,1956],{},[1947,1954,1955],{},"Relying only on a blank row."," Total rows often follow the data directly; check the key's format too.",[1944,1958,1959,1962],{},[1947,1960,1961],{},"Throwing the notes away."," Keep source and caveats as metadata.",[1944,1964,1965,1968],{},[1947,1966,1967],{},"Leaving marked values as text."," Split the marker into a flag and convert the number.",[1944,1970,1971,1974],{},[1947,1972,1973],{},"Not using the report's own totals."," They are a free check on the extraction.",[29,1976,1978],{"id":1977},"conclusion","Conclusion",[10,1980,1981,1982,1984,1985,1988],{},"To read only the data from a report sheet with totals and notes underneath, find the last data row first — the first blank row, the first row whose key does not match the expected pattern, or the first footer word — and pass ",[14,1983,1090],{}," to ",[14,1986,1987],{},"read_excel",". Capture the lines below as metadata, because sources and caveats belong with the data. Split footnote markers such as a trailing asterisk into a clean number and a flag column, and confirm the extraction by comparing your totals with the report's own total row.",[29,1990,1992],{"id":1991},"frequently-asked-questions","Frequently asked questions",[10,1994,1995,1998],{},[1947,1996,1997],{},"How do I stop pandas reading the notes below an Excel table?","\nFind the last data row, for example the row before the first blank row or the first row whose key column does not match the expected format, and pass nrows to read_excel.",[10,2000,2001,2004],{},[1947,2002,2003],{},"How do I exclude the total row when reading an Excel report?","\nStop reading before it by detecting a label such as Total, or a key that does not match the data's pattern, then read with nrows. Keep the total for checking your sums.",[10,2006,2007,2010],{},[1947,2008,2009],{},"How do I keep the footnotes from an Excel sheet?","\nRead the cells below the last data row with openpyxl and store them, for example as a dictionary of source and notes, alongside the data.",[10,2012,2013,2016],{},[1947,2014,2015],{},"How do I clean numbers that have footnote markers like an asterisk?","\nStrip the marker, remove thousands separators and convert to a number, and record whether the marker was present in a separate boolean column.",[29,2018,2020],{"id":2019},"related","Related",[1941,2022,2023,2029,2036,2043],{},[1944,2024,2025,2026,2028],{},"Up one level: ",[23,2027,26],{"href":25}," — getting data out of awkward layouts.",[1944,2030,2031,2035],{},[23,2032,2034],{"href":2033},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Ffind-the-header-row-in-an-excel-sheet-with-a-shifting-layout\u002F","Find the Header Row in an Excel Sheet with a Shifting Layout"," — the top edge of the same problem.",[1944,2037,2038,2042],{},[23,2039,2041],{"href":2040},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fread-multiple-tables-from-one-excel-sheet\u002F","Read Multiple Tables from One Excel Sheet"," — when the next block is another table.",[1944,2044,2045,2049],{},[23,2046,2048],{"href":2047},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fconvert-currency-strings-to-numbers-with-pandas\u002F","Convert Currency Strings to Numbers with pandas"," — more number cleaning.",[2051,2052,2053],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}html pre.shiki code .s-wDw, html code.shiki .s-wDw{--shiki-default:#6A737D;--shiki-dark:#BDC4CC}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}",{"title":39,"searchDepth":96,"depth":96,"links":2055},[2056,2057,2058,2059,2060,2061,2062,2063,2064,2065],{"id":31,"depth":96,"text":32},{"id":69,"depth":96,"text":70},{"id":184,"depth":96,"text":185},{"id":1083,"depth":96,"text":1084},{"id":1187,"depth":96,"text":1188},{"id":1523,"depth":96,"text":1524},{"id":1938,"depth":96,"text":1939},{"id":1977,"depth":96,"text":1978},{"id":1991,"depth":96,"text":1992},{"id":2019,"depth":96,"text":2020},"2026-10-02","Many Excel reports put a total row, a blank line and footnotes such as \"Source:\" or \"* provisional\" under the table, and pandas reads them all as data. Find where the table really ends — the first blank row, the first row whose key does not look like a key, or a footer marker — read exactly that many rows, keep the footnotes as metadata, and turn footnote markers inside cells into a flag column.","md",[2070,2072,2074,2076],{"q":1997,"a":2071},"Find the last data row, for example the row before the first blank row or the first row whose key column does not match the expected format, and pass nrows to read_excel.",{"q":2003,"a":2073},"Stop reading before it by detecting a label such as Total, or a key that does not match the data's pattern, then read with nrows. Keep the total for checking your sums.",{"q":2009,"a":2075},"Read the cells below the last data row with openpyxl and store them, for example as a dictionary of source and notes, alongside the data.",{"q":2015,"a":2077},"Strip the marker, remove thousands separators and convert to a number, and record whether the marker was present in a separate boolean column.","stop reading at notes and footers below the data",null,{"breadcrumb":2081},[2082,2084,2087],{"name":2083,"item":1731},"Home",{"name":2085,"item":2086},"Advanced Data Transformation and Cleaning for Excel Automation","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":26,"item":25},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data",{"title":2090,"description":2091},"Stop Reading Excel at Notes and Footers Below the Data","Exclude total rows, blank lines and footnotes below an Excel table: detect the end of the data, read with nrows, capture footnotes as metadata, and convert footnote markers like 123* into clean numbers plus a flag.","stop-reading-at-notes-and-footers-below-the-data","advanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fstop-reading-at-notes-and-footers-below-the-data\u002Findex",[2095,2096],"messy-sheets","reading-excel","how-to","7B9tOTqqnPojCXiFG_L3D_K0_-jnk-QYRN0jzRIQxGI",[2100,2104],{"title":2101,"path":2102,"stem":2103,"children":-1},"Skip Hidden and Filtered Rows When Reading Excel","\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fskip-hidden-and-filtered-rows-when-reading-excel","advanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fskip-hidden-and-filtered-rows-when-reading-excel\u002Findex",{"title":2105,"path":2106,"stem":2107,"children":-1},"Turn a Pivot Table Export Back into Tidy Data","\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data","advanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fturn-a-pivot-table-export-back-into-tidy-data\u002Findex",[2109,2111],{"path":2110,"title":26},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets",{"path":2112,"title":2085},"\u002Fadvanced-data-transformation-and-cleaning",[],[2115,2118,2121,2124,2127,2130],{"path":2116,"title":2117,"kind":2097},"\u002Fformatting-and-charting-excel-reports-with-python\u002Fstyling-excel-cells-with-openpyxl\u002Ffreeze-header-row-in-excel-with-openpyxl","Freeze the Header Row in Excel with openpyxl",{"path":2119,"title":2120,"kind":2097},"\u002Fgetting-started-with-python-excel-automation\u002Freading-excel-files-with-pandas\u002Fhandle-merged-cells-when-reading-excel-with-pandas","Handle Merged Cells When Reading Excel with pandas",{"path":2122,"title":2123,"kind":2097},"\u002Fgetting-started-with-python-excel-automation\u002Fusing-openpyxl-for-excel-file-manipulation\u002Funmerge-cells-and-fill-values-with-openpyxl","Unmerge Cells and Fill the Values with openpyxl",{"path":2125,"title":2126,"kind":2097},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fcombine-sheets-with-different-layouts-into-one-table","Combine Sheets with Different Layouts into One Table",{"path":2128,"title":2129,"kind":2097},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fextract-hyperlink-urls-from-excel-cells","Extract Hyperlink URLs from Excel Cells with openpyxl",{"path":2131,"title":2132,"kind":2097},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fextract-report-header-fields-from-an-excel-sheet","Extract Report Header Fields from an Excel Sheet with Python",1790972094311]