[{"data":1,"prerenderedAt":2207},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts":2173,"ancestors:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts":2182,"children:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts":2187,"related:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts":2188},{"id":4,"title":5,"body":6,"dateModified":2138,"datePublished":2138,"description":2139,"extension":2140,"faq":2141,"keyword":2150,"kind":2151,"meta":2152,"navigation":109,"path":2162,"seo":2163,"slug":2166,"stem":2167,"topics":2168,"type":2171,"__hash__":2172},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts\u002Findex.md","How Missing Values Change Sums, Averages and Counts",{"type":7,"value":8,"toc":2125},"minimark",[9,19,24,53,60,64,67,535,542,546,549,768,807,979,983,986,1178,1231,1234,1360,1385,1389,1392,1493,1538,1624,1628,1631,1732,1744,1748,1751,1833,1849,1970,1974,2024,2028,2060,2064,2070,2076,2082,2088,2092,2121],[10,11,12,13,18],"p",{},"Two people summarise the same Excel sheet — one with formulas, one with pandas — and get different averages. Neither made a mistake: they treated the blank cells differently. Missing values are invisible in a total but change almost every other summary: an average over five filled cells is not an average over eight, a count of rows is not a count of values, and a group with no values at all can show a sum of zero that looks like a real result. This page sets out what each common Excel function and its pandas counterpart does with blanks, zeros and text placeholders such as \"N\u002FA\", checks the Excel side by recalculating real formulas, and ends with a habit that keeps both tools in agreement. It belongs to ",[14,15,17],"a",{"href":16},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002F","Handling Missing Data in Excel Reports",".",[20,21,23],"h2",{"id":22},"prerequisites","Prerequisites",[25,26,31],"pre",{"className":27,"code":28,"language":29,"meta":30,"style":30},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash","",[32,33,34],"code",{"__ignoreMap":30},[35,36,39,43,47,50],"span",{"class":37,"line":38},"line",1,[35,40,42],{"class":41},"sMTad","pip",[35,44,46],{"class":45},"srMev"," install",[35,48,49],{"class":45}," pandas",[35,51,52],{"class":45}," openpyxl\n",[10,54,55,56,59],{},"The examples build a small workbook of monthly sales by rep with a genuine blank, a zero and an \"N\u002FA\" text placeholder, write Excel formulas next to it, and recalculate them with LibreOffice (",[32,57,58],{},"soffice",") so the Excel results can be compared with pandas. Any spreadsheet that recalculates gives the same answers for these functions.",[20,61,63],{"id":62},"step-1-a-sheet-with-blanks-zeros-and-placeholders","Step 1 — A sheet with blanks, zeros and placeholders",[10,65,66],{},"Real exports mix three different things in the same column: real zeros, empty cells, and text standing in for \"unknown\":",[25,68,72],{"className":69,"code":70,"language":71,"meta":30,"style":30},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\nfrom openpyxl import Workbook\n\nrows = [(\"North\", \"Ana\", 1200), (\"North\", \"Ben\", None), (\"North\", \"Chloé\", 0),\n        (\"South\", \"Dev\", 950), (\"South\", \"Eva\", \"N\u002FA\"), (\"South\", \"Farid\", 1100),\n        (\"West\", \"Grace\", None), (\"West\", \"Hugo\", None)]\n\nwb = Workbook()\nws = wb.active\nws.title = \"Sales\"\nws.append([\"Region\", \"Rep\", \"Sales\"])\nfor r in rows:\n    ws.append(list(r))\nlabels = [\"SUM\", \"AVERAGE\", \"COUNT\", \"COUNTA\", \"COUNTBLANK\", \"ROWS\"]\nformulas = [\"=SUM(C2:C9)\", \"=AVERAGE(C2:C9)\", \"=COUNT(C2:C9)\", \"=COUNTA(C2:C9)\", \"=COUNTBLANK(C2:C9)\", \"=ROWS(C2:C9)\"]\nfor i, (lab, f) in enumerate(zip(labels, formulas), start=2):\n    ws.cell(i, 5, lab)\n    ws.cell(i, 6, f)\nws[\"E9\"] = \"AVERAGEIF West\"\nws[\"F9\"] = '=IFERROR(AVERAGEIF(A2:A9,\"West\",C2:C9),\"#DIV\u002F0!\")'\nwb.save(\"sales.xlsx\")\nprint(\"written: 8 reps, 3 blank, 1 zero, 1 'N\u002FA'\")\n","python",[32,73,74,90,104,111,170,219,252,257,268,279,290,312,327,339,381,421,455,467,478,495,510,522],{"__ignoreMap":30},[35,75,76,80,84,87],{"class":37,"line":38},[35,77,79],{"class":78},"s-kum","import",[35,81,83],{"class":82},"skGVy"," pandas ",[35,85,86],{"class":78},"as",[35,88,89],{"class":82}," pd\n",[35,91,93,96,99,101],{"class":37,"line":92},2,[35,94,95],{"class":78},"from",[35,97,98],{"class":82}," openpyxl ",[35,100,79],{"class":78},[35,102,103],{"class":82}," Workbook\n",[35,105,107],{"class":37,"line":106},3,[35,108,110],{"emptyLinePlaceholder":109},true,"\n",[35,112,114,117,120,123,126,129,132,134,138,141,143,145,148,150,153,155,157,159,162,164,167],{"class":37,"line":113},4,[35,115,116],{"class":82},"rows ",[35,118,119],{"class":78},"=",[35,121,122],{"class":82}," [(",[35,124,125],{"class":45},"\"North\"",[35,127,128],{"class":82},", ",[35,130,131],{"class":45},"\"Ana\"",[35,133,128],{"class":82},[35,135,137],{"class":136},"sP0c6","1200",[35,139,140],{"class":82},"), (",[35,142,125],{"class":45},[35,144,128],{"class":82},[35,146,147],{"class":45},"\"Ben\"",[35,149,128],{"class":82},[35,151,152],{"class":136},"None",[35,154,140],{"class":82},[35,156,125],{"class":45},[35,158,128],{"class":82},[35,160,161],{"class":45},"\"Chloé\"",[35,163,128],{"class":82},[35,165,166],{"class":136},"0",[35,168,169],{"class":82},"),\n",[35,171,173,176,179,181,184,186,189,191,193,195,198,200,203,205,207,209,212,214,217],{"class":37,"line":172},5,[35,174,175],{"class":82},"        (",[35,177,178],{"class":45},"\"South\"",[35,180,128],{"class":82},[35,182,183],{"class":45},"\"Dev\"",[35,185,128],{"class":82},[35,187,188],{"class":136},"950",[35,190,140],{"class":82},[35,192,178],{"class":45},[35,194,128],{"class":82},[35,196,197],{"class":45},"\"Eva\"",[35,199,128],{"class":82},[35,201,202],{"class":45},"\"N\u002FA\"",[35,204,140],{"class":82},[35,206,178],{"class":45},[35,208,128],{"class":82},[35,210,211],{"class":45},"\"Farid\"",[35,213,128],{"class":82},[35,215,216],{"class":136},"1100",[35,218,169],{"class":82},[35,220,222,224,227,229,232,234,236,238,240,242,245,247,249],{"class":37,"line":221},6,[35,223,175],{"class":82},[35,225,226],{"class":45},"\"West\"",[35,228,128],{"class":82},[35,230,231],{"class":45},"\"Grace\"",[35,233,128],{"class":82},[35,235,152],{"class":136},[35,237,140],{"class":82},[35,239,226],{"class":45},[35,241,128],{"class":82},[35,243,244],{"class":45},"\"Hugo\"",[35,246,128],{"class":82},[35,248,152],{"class":136},[35,250,251],{"class":82},")]\n",[35,253,255],{"class":37,"line":254},7,[35,256,110],{"emptyLinePlaceholder":109},[35,258,260,263,265],{"class":37,"line":259},8,[35,261,262],{"class":82},"wb ",[35,264,119],{"class":78},[35,266,267],{"class":82}," Workbook()\n",[35,269,271,274,276],{"class":37,"line":270},9,[35,272,273],{"class":82},"ws ",[35,275,119],{"class":78},[35,277,278],{"class":82}," wb.active\n",[35,280,282,285,287],{"class":37,"line":281},10,[35,283,284],{"class":82},"ws.title ",[35,286,119],{"class":78},[35,288,289],{"class":45}," \"Sales\"\n",[35,291,293,296,299,301,304,306,309],{"class":37,"line":292},11,[35,294,295],{"class":82},"ws.append([",[35,297,298],{"class":45},"\"Region\"",[35,300,128],{"class":82},[35,302,303],{"class":45},"\"Rep\"",[35,305,128],{"class":82},[35,307,308],{"class":45},"\"Sales\"",[35,310,311],{"class":82},"])\n",[35,313,315,318,321,324],{"class":37,"line":314},12,[35,316,317],{"class":78},"for",[35,319,320],{"class":82}," r ",[35,322,323],{"class":78},"in",[35,325,326],{"class":82}," rows:\n",[35,328,330,333,336],{"class":37,"line":329},13,[35,331,332],{"class":82},"    ws.append(",[35,334,335],{"class":136},"list",[35,337,338],{"class":82},"(r))\n",[35,340,342,345,347,350,353,355,358,360,363,365,368,370,373,375,378],{"class":37,"line":341},14,[35,343,344],{"class":82},"labels ",[35,346,119],{"class":78},[35,348,349],{"class":82}," [",[35,351,352],{"class":45},"\"SUM\"",[35,354,128],{"class":82},[35,356,357],{"class":45},"\"AVERAGE\"",[35,359,128],{"class":82},[35,361,362],{"class":45},"\"COUNT\"",[35,364,128],{"class":82},[35,366,367],{"class":45},"\"COUNTA\"",[35,369,128],{"class":82},[35,371,372],{"class":45},"\"COUNTBLANK\"",[35,374,128],{"class":82},[35,376,377],{"class":45},"\"ROWS\"",[35,379,380],{"class":82},"]\n",[35,382,384,387,389,391,394,396,399,401,404,406,409,411,414,416,419],{"class":37,"line":383},15,[35,385,386],{"class":82},"formulas ",[35,388,119],{"class":78},[35,390,349],{"class":82},[35,392,393],{"class":45},"\"=SUM(C2:C9)\"",[35,395,128],{"class":82},[35,397,398],{"class":45},"\"=AVERAGE(C2:C9)\"",[35,400,128],{"class":82},[35,402,403],{"class":45},"\"=COUNT(C2:C9)\"",[35,405,128],{"class":82},[35,407,408],{"class":45},"\"=COUNTA(C2:C9)\"",[35,410,128],{"class":82},[35,412,413],{"class":45},"\"=COUNTBLANK(C2:C9)\"",[35,415,128],{"class":82},[35,417,418],{"class":45},"\"=ROWS(C2:C9)\"",[35,420,380],{"class":82},[35,422,424,426,429,431,434,437,440,443,447,449,452],{"class":37,"line":423},16,[35,425,317],{"class":78},[35,427,428],{"class":82}," i, (lab, f) ",[35,430,323],{"class":78},[35,432,433],{"class":136}," enumerate",[35,435,436],{"class":82},"(",[35,438,439],{"class":136},"zip",[35,441,442],{"class":82},"(labels, formulas), ",[35,444,446],{"class":445},"sa561","start",[35,448,119],{"class":78},[35,450,451],{"class":136},"2",[35,453,454],{"class":82},"):\n",[35,456,458,461,464],{"class":37,"line":457},17,[35,459,460],{"class":82},"    ws.cell(i, ",[35,462,463],{"class":136},"5",[35,465,466],{"class":82},", lab)\n",[35,468,470,472,475],{"class":37,"line":469},18,[35,471,460],{"class":82},[35,473,474],{"class":136},"6",[35,476,477],{"class":82},", f)\n",[35,479,481,484,487,490,492],{"class":37,"line":480},19,[35,482,483],{"class":82},"ws[",[35,485,486],{"class":45},"\"E9\"",[35,488,489],{"class":82},"] ",[35,491,119],{"class":78},[35,493,494],{"class":45}," \"AVERAGEIF West\"\n",[35,496,498,500,503,505,507],{"class":37,"line":497},20,[35,499,483],{"class":82},[35,501,502],{"class":45},"\"F9\"",[35,504,489],{"class":82},[35,506,119],{"class":78},[35,508,509],{"class":45}," '=IFERROR(AVERAGEIF(A2:A9,\"West\",C2:C9),\"#DIV\u002F0!\")'\n",[35,511,513,516,519],{"class":37,"line":512},21,[35,514,515],{"class":82},"wb.save(",[35,517,518],{"class":45},"\"sales.xlsx\"",[35,520,521],{"class":82},")\n",[35,523,525,528,530,533],{"class":37,"line":524},22,[35,526,527],{"class":136},"print",[35,529,436],{"class":82},[35,531,532],{"class":45},"\"written: 8 reps, 3 blank, 1 zero, 1 'N\u002FA'\"",[35,534,521],{"class":82},[10,536,537,538,541],{},"Ben, Grace and Hugo have no sales figure at all; Chloé genuinely sold nothing; Eva's figure is the text \"N\u002FA\". Formulas in column F summarise the column the way most spreadsheets do. In a database, or in pandas after reading, these distinctions survive only if you keep them — a blank becomes ",[32,539,540],{},"NaN",", a zero stays 0, and \"N\u002FA\" stays a string unless you convert it.",[20,543,545],{"id":544},"step-2-what-excels-functions-do","Step 2 — What Excel's functions do",[10,547,548],{},"Recalculate the formulas and read back the results:",[25,550,552],{"className":69,"code":551,"language":71,"meta":30,"style":30},"import subprocess\nfrom openpyxl import load_workbook\n\nsubprocess.run([\"soffice\", \"--headless\", \"--convert-to\", \"xlsx\", \"--outdir\", \"calc\", \"sales.xlsx\"],\n               check=True, capture_output=True)\ncalc = load_workbook(\"calc\u002Fsales.xlsx\", data_only=True)[\"Sales\"]\nexcel = {calc.cell(r, 5).value: calc.cell(r, 6).value for r in range(2, 10) if calc.cell(r, 5).value}\nfor k, v in excel.items():\n    print(f\"{k:16} {v}\")\n",[32,553,554,561,572,576,616,637,666,718,730],{"__ignoreMap":30},[35,555,556,558],{"class":37,"line":38},[35,557,79],{"class":78},[35,559,560],{"class":82}," subprocess\n",[35,562,563,565,567,569],{"class":37,"line":92},[35,564,95],{"class":78},[35,566,98],{"class":82},[35,568,79],{"class":78},[35,570,571],{"class":82}," load_workbook\n",[35,573,574],{"class":37,"line":106},[35,575,110],{"emptyLinePlaceholder":109},[35,577,578,581,584,586,589,591,594,596,599,601,604,606,609,611,613],{"class":37,"line":113},[35,579,580],{"class":82},"subprocess.run([",[35,582,583],{"class":45},"\"soffice\"",[35,585,128],{"class":82},[35,587,588],{"class":45},"\"--headless\"",[35,590,128],{"class":82},[35,592,593],{"class":45},"\"--convert-to\"",[35,595,128],{"class":82},[35,597,598],{"class":45},"\"xlsx\"",[35,600,128],{"class":82},[35,602,603],{"class":45},"\"--outdir\"",[35,605,128],{"class":82},[35,607,608],{"class":45},"\"calc\"",[35,610,128],{"class":82},[35,612,518],{"class":45},[35,614,615],{"class":82},"],\n",[35,617,618,621,623,626,628,631,633,635],{"class":37,"line":172},[35,619,620],{"class":445},"               check",[35,622,119],{"class":78},[35,624,625],{"class":136},"True",[35,627,128],{"class":82},[35,629,630],{"class":445},"capture_output",[35,632,119],{"class":78},[35,634,625],{"class":136},[35,636,521],{"class":82},[35,638,639,642,644,647,650,652,655,657,659,662,664],{"class":37,"line":221},[35,640,641],{"class":82},"calc ",[35,643,119],{"class":78},[35,645,646],{"class":82}," load_workbook(",[35,648,649],{"class":45},"\"calc\u002Fsales.xlsx\"",[35,651,128],{"class":82},[35,653,654],{"class":445},"data_only",[35,656,119],{"class":78},[35,658,625],{"class":136},[35,660,661],{"class":82},")[",[35,663,308],{"class":45},[35,665,380],{"class":82},[35,667,668,671,673,676,678,681,683,686,688,690,692,695,697,699,701,704,707,710,713,715],{"class":37,"line":254},[35,669,670],{"class":82},"excel ",[35,672,119],{"class":78},[35,674,675],{"class":82}," {calc.cell(r, ",[35,677,463],{"class":136},[35,679,680],{"class":82},").value: calc.cell(r, ",[35,682,474],{"class":136},[35,684,685],{"class":82},").value ",[35,687,317],{"class":78},[35,689,320],{"class":82},[35,691,323],{"class":78},[35,693,694],{"class":136}," range",[35,696,436],{"class":82},[35,698,451],{"class":136},[35,700,128],{"class":82},[35,702,703],{"class":136},"10",[35,705,706],{"class":82},") ",[35,708,709],{"class":78},"if",[35,711,712],{"class":82}," calc.cell(r, ",[35,714,463],{"class":136},[35,716,717],{"class":82},").value}\n",[35,719,720,722,725,727],{"class":37,"line":259},[35,721,317],{"class":78},[35,723,724],{"class":82}," k, v ",[35,726,323],{"class":78},[35,728,729],{"class":82}," excel.items():\n",[35,731,732,735,737,740,743,747,750,753,756,759,762,764,766],{"class":37,"line":270},[35,733,734],{"class":136},"    print",[35,736,436],{"class":82},[35,738,739],{"class":78},"f",[35,741,742],{"class":45},"\"",[35,744,746],{"class":745},"sSjpA","{",[35,748,749],{"class":82},"k",[35,751,752],{"class":78},":16",[35,754,755],{"class":745},"}",[35,757,758],{"class":745}," {",[35,760,761],{"class":82},"v",[35,763,755],{"class":745},[35,765,742],{"class":45},[35,767,521],{"class":82},[10,769,770,773,774,777,778,782,783,786,787,790,791,794,795,798,799,802,803,806],{},[32,771,772],{},"SUM"," adds the three numbers (1,200 + 0 + 950 + 1,100 = 3,250) and ignores both blanks and the text. ",[32,775,776],{},"AVERAGE"," divides by the count of ",[779,780,781],"em",{},"numbers"," — four, including Chloé's zero — so 812.50; blanks and \"N\u002FA\" do not count, but the zero does. ",[32,784,785],{},"COUNT"," counts numeric cells (4), ",[32,788,789],{},"COUNTA"," counts non-empty cells including text (5), and ",[32,792,793],{},"COUNTBLANK"," the empty ones (3). ",[32,796,797],{},"ROWS"," counts all eight. For the West region, where every value is blank, ",[32,800,801],{},"AVERAGEIF"," has nothing to average and returns ",[32,804,805],{},"#DIV\u002F0!"," — Excel refuses to invent a figure. These rules are worth knowing exactly, because pandas follows some of them and not others.",[808,809,817,818,817,822,817,826,817,833,817,841,817,848,817,854,817,857,817,860,817,863,817,866,817,869,817,872,817,877,817,883,817,885,817,890,817,892,817,895,817,897,817,900,817,902,817,904,817,907,817,911,817,913,817,917,817,919,817,922,817,924,817,926,817,928,817,930,817,933,817,937,817,939,817,943,817,945,817,947,817,949,817,951,817,953,817,955,817,958,817,962,817,964,817,967,817,969,817,971,817,973,817,975,817,977],"svg",{"viewBox":810,"role":811,"ariaLabelledBy":812,"xmlns":815,"style":816},"4 0 752 208","img",[813,814],"mv-excel-t","mv-excel-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[819,820,821],"title",{"id":813},"How each cell kind is treated",[823,824,825],"desc",{"id":814},"A number is included by SUM, AVERAGE, COUNT and COUNTA. A zero is a number and is included everywhere, lowering the average. A blank is ignored by SUM, AVERAGE, COUNT and COUNTA and counted by COUNTBLANK. Text such as N\u002FA is ignored by SUM, AVERAGE and COUNT but counted by COUNTA.",[827,828],"rect",{"x":829,"y":166,"width":830,"height":831,"fill":832},"4","752","208","#ffffff",[827,834],{"x":835,"y":836,"width":837,"height":838,"rx":166,"fill":832,"stroke":839,"style":840},"20.0","16.0","126.8","40.0","var(--line,#cdd5e6)","stroke-width:1px",[827,842],{"x":843,"y":836,"width":844,"height":838,"rx":166,"fill":845,"stroke":846,"style":847},"146.8","148.3","#f0f4ff","var(--brand,#5b5cf0)","stroke-width:1.2px",[849,850,772],"text",{"x":851,"y":852,"style":853},"220.9","41.0","font-size:13px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle",[827,855],{"x":856,"y":836,"width":844,"height":838,"rx":166,"fill":845,"stroke":846,"style":847},"295.1",[849,858,776],{"x":859,"y":852,"style":853},"369.2",[827,861],{"x":862,"y":836,"width":844,"height":838,"rx":166,"fill":845,"stroke":846,"style":847},"443.4",[849,864,785],{"x":865,"y":852,"style":853},"517.5",[827,867],{"x":868,"y":836,"width":844,"height":838,"rx":166,"fill":845,"stroke":846,"style":847},"591.7",[849,870,789],{"x":871,"y":852,"style":853},"665.8",[827,873],{"x":835,"y":874,"width":837,"height":875,"rx":166,"fill":876,"stroke":839,"style":840},"56.0","34.0","#e7ebef",[849,878,882],{"x":879,"y":880,"style":881},"32.0","78.0","font-size:12.5px;font-weight:700;fill:var(--text,#172033);text-anchor:start","number (1,200)",[827,884],{"x":843,"y":874,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,886,889],{"x":851,"y":887,"style":888},"77.0","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","added",[827,891],{"x":856,"y":874,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,893,894],{"x":859,"y":887,"style":888},"included",[827,896],{"x":862,"y":874,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,898,899],{"x":865,"y":887,"style":888},"counted",[827,901],{"x":868,"y":874,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,903,899],{"x":871,"y":887,"style":888},[827,905],{"x":835,"y":906,"width":837,"height":875,"rx":166,"fill":876,"stroke":839,"style":840},"90.0",[849,908,910],{"x":879,"y":909,"style":881},"112.0","zero (0)",[827,912],{"x":843,"y":906,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,914,916],{"x":851,"y":915,"style":888},"111.0","adds 0",[827,918],{"x":856,"y":906,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,920,921],{"x":859,"y":915,"style":888},"included ↓",[827,923],{"x":862,"y":906,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,925,899],{"x":865,"y":915,"style":888},[827,927],{"x":868,"y":906,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,929,899],{"x":871,"y":915,"style":888},[827,931],{"x":835,"y":932,"width":837,"height":875,"rx":166,"fill":876,"stroke":839,"style":840},"124.0",[849,934,936],{"x":879,"y":935,"style":881},"146.0","blank",[827,938],{"x":843,"y":932,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,940,942],{"x":851,"y":941,"style":888},"145.0","ignored",[827,944],{"x":856,"y":932,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,946,942],{"x":859,"y":941,"style":888},[827,948],{"x":862,"y":932,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,950,942],{"x":865,"y":941,"style":888},[827,952],{"x":868,"y":932,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,954,942],{"x":871,"y":941,"style":888},[827,956],{"x":835,"y":957,"width":837,"height":875,"rx":166,"fill":876,"stroke":839,"style":840},"158.0",[849,959,961],{"x":879,"y":960,"style":881},"180.0","text (N\u002FA)",[827,963],{"x":843,"y":957,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,965,942],{"x":851,"y":966,"style":888},"179.0",[827,968],{"x":856,"y":957,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,970,942],{"x":859,"y":966,"style":888},[827,972],{"x":862,"y":957,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,974,942],{"x":865,"y":966,"style":888},[827,976],{"x":868,"y":957,"width":844,"height":875,"rx":166,"fill":832,"stroke":839,"style":840},[849,978,899],{"x":871,"y":966,"style":888},[20,980,982],{"id":981},"step-3-the-pandas-equivalents-and-a-surprise-at-read-time","Step 3 — The pandas equivalents, and a surprise at read time",[10,984,985],{},"Read the same sheet with pandas and compute the matching figures:",[25,987,989],{"className":69,"code":988,"language":71,"meta":30,"style":30},"df = pd.read_excel(\"sales.xlsx\", usecols=\"A:C\")\nprint(df[\"Sales\"].tolist(), df[\"Sales\"].dtype)\n\nsales = df[\"Sales\"]\npandas_side = {\n    \"SUM\": sales.sum(),\n    \"AVERAGE\": sales.mean(),\n    \"COUNT\": sales.count(),\n    \"COUNTA\": sales.count(),\n    \"COUNTBLANK\": int(sales.isna().sum()),\n    \"ROWS\": len(df),\n}\nfor k, v in pandas_side.items():\n    print(f\"{k:12} pandas {v!s:>8}   Excel {excel[k]}\")\n",[32,990,991,1015,1032,1036,1050,1060,1068,1076,1084,1091,1105,1118,1123,1134],{"__ignoreMap":30},[35,992,993,996,998,1001,1003,1005,1008,1010,1013],{"class":37,"line":38},[35,994,995],{"class":82},"df ",[35,997,119],{"class":78},[35,999,1000],{"class":82}," pd.read_excel(",[35,1002,518],{"class":45},[35,1004,128],{"class":82},[35,1006,1007],{"class":445},"usecols",[35,1009,119],{"class":78},[35,1011,1012],{"class":45},"\"A:C\"",[35,1014,521],{"class":82},[35,1016,1017,1019,1022,1024,1027,1029],{"class":37,"line":92},[35,1018,527],{"class":136},[35,1020,1021],{"class":82},"(df[",[35,1023,308],{"class":45},[35,1025,1026],{"class":82},"].tolist(), df[",[35,1028,308],{"class":45},[35,1030,1031],{"class":82},"].dtype)\n",[35,1033,1034],{"class":37,"line":106},[35,1035,110],{"emptyLinePlaceholder":109},[35,1037,1038,1041,1043,1046,1048],{"class":37,"line":113},[35,1039,1040],{"class":82},"sales ",[35,1042,119],{"class":78},[35,1044,1045],{"class":82}," df[",[35,1047,308],{"class":45},[35,1049,380],{"class":82},[35,1051,1052,1055,1057],{"class":37,"line":172},[35,1053,1054],{"class":82},"pandas_side ",[35,1056,119],{"class":78},[35,1058,1059],{"class":82}," {\n",[35,1061,1062,1065],{"class":37,"line":221},[35,1063,1064],{"class":45},"    \"SUM\"",[35,1066,1067],{"class":82},": sales.sum(),\n",[35,1069,1070,1073],{"class":37,"line":254},[35,1071,1072],{"class":45},"    \"AVERAGE\"",[35,1074,1075],{"class":82},": sales.mean(),\n",[35,1077,1078,1081],{"class":37,"line":259},[35,1079,1080],{"class":45},"    \"COUNT\"",[35,1082,1083],{"class":82},": sales.count(),\n",[35,1085,1086,1089],{"class":37,"line":270},[35,1087,1088],{"class":45},"    \"COUNTA\"",[35,1090,1083],{"class":82},[35,1092,1093,1096,1099,1102],{"class":37,"line":281},[35,1094,1095],{"class":45},"    \"COUNTBLANK\"",[35,1097,1098],{"class":82},": ",[35,1100,1101],{"class":136},"int",[35,1103,1104],{"class":82},"(sales.isna().sum()),\n",[35,1106,1107,1110,1112,1115],{"class":37,"line":292},[35,1108,1109],{"class":45},"    \"ROWS\"",[35,1111,1098],{"class":82},[35,1113,1114],{"class":136},"len",[35,1116,1117],{"class":82},"(df),\n",[35,1119,1120],{"class":37,"line":314},[35,1121,1122],{"class":82},"}\n",[35,1124,1125,1127,1129,1131],{"class":37,"line":329},[35,1126,317],{"class":78},[35,1128,724],{"class":82},[35,1130,323],{"class":78},[35,1132,1133],{"class":82}," pandas_side.items():\n",[35,1135,1136,1138,1140,1142,1144,1146,1148,1151,1153,1155,1157,1159,1162,1164,1167,1169,1172,1174,1176],{"class":37,"line":341},[35,1137,734],{"class":136},[35,1139,436],{"class":82},[35,1141,739],{"class":78},[35,1143,742],{"class":45},[35,1145,746],{"class":745},[35,1147,749],{"class":82},[35,1149,1150],{"class":78},":12",[35,1152,755],{"class":745},[35,1154,83],{"class":45},[35,1156,746],{"class":745},[35,1158,761],{"class":82},[35,1160,1161],{"class":78},"!s:>8",[35,1163,755],{"class":745},[35,1165,1166],{"class":45},"   Excel ",[35,1168,746],{"class":745},[35,1170,1171],{"class":82},"excel[k]",[35,1173,755],{"class":745},[35,1175,742],{"class":45},[35,1177,521],{"class":82},[10,1179,1180,1183,1184,1187,1188,1191,1192,128,1194,1196,1197,1199,1200,1203,1204,1207,1208,1210,1211,1196,1213,1215,1216,1219,1220,1222,1223,1226,1227,1230],{},[32,1181,1182],{},"sum()"," skips missing values, ",[32,1185,1186],{},"mean()"," divides by the number of non-missing values and ",[32,1189,1190],{},"count()"," counts non-missing values, so they match Excel's ",[32,1193,772],{},[32,1195,776],{}," and ",[32,1198,785],{},"; ",[32,1201,1202],{},"len(df)"," — or ",[32,1205,1206],{},"size"," in a groupby — counts rows like ",[32,1209,797],{},". But ",[32,1212,789],{},[32,1214,793],{}," disagree: pandas says 4 and 4, Excel 5 and 3. The column came back as ",[32,1217,1218],{},"float64",", and Eva's \"N\u002FA\" is ",[32,1221,540],{},". That is pandas' default behaviour: ",[32,1224,1225],{},"read_excel"," treats a list of common placeholder strings — \"N\u002FA\", \"NA\", \"n\u002Fa\", \"NULL\", \"nan\", \"#N\u002FA\" and others — as missing ",[779,1228,1229],{},"while reading",". Usually that is convenient; here it erased the difference between \"the cell says N\u002FA\" and \"the cell is empty\", which Excel keeps.",[10,1232,1233],{},"To count the way Excel does, read the column with the default placeholders switched off, keeping only truly empty cells as missing:",[25,1235,1237],{"className":69,"code":1236,"language":71,"meta":30,"style":30},"raw = pd.read_excel(\"sales.xlsx\", usecols=\"A:C\", keep_default_na=False, na_values=[\"\"])\nprint(raw[\"Sales\"].tolist())\npandas_side[\"COUNTA\"] = int(raw[\"Sales\"].count())\npandas_side[\"COUNTBLANK\"] = int(raw[\"Sales\"].isna().sum())\nprint(\"COUNTA\", pandas_side[\"COUNTA\"], \"| COUNTBLANK\", pandas_side[\"COUNTBLANK\"])\n",[32,1238,1239,1283,1295,1316,1335],{"__ignoreMap":30},[35,1240,1241,1244,1246,1248,1250,1252,1254,1256,1258,1260,1263,1265,1268,1270,1273,1275,1278,1281],{"class":37,"line":38},[35,1242,1243],{"class":82},"raw ",[35,1245,119],{"class":78},[35,1247,1000],{"class":82},[35,1249,518],{"class":45},[35,1251,128],{"class":82},[35,1253,1007],{"class":445},[35,1255,119],{"class":78},[35,1257,1012],{"class":45},[35,1259,128],{"class":82},[35,1261,1262],{"class":445},"keep_default_na",[35,1264,119],{"class":78},[35,1266,1267],{"class":136},"False",[35,1269,128],{"class":82},[35,1271,1272],{"class":445},"na_values",[35,1274,119],{"class":78},[35,1276,1277],{"class":82},"[",[35,1279,1280],{"class":45},"\"\"",[35,1282,311],{"class":82},[35,1284,1285,1287,1290,1292],{"class":37,"line":92},[35,1286,527],{"class":136},[35,1288,1289],{"class":82},"(raw[",[35,1291,308],{"class":45},[35,1293,1294],{"class":82},"].tolist())\n",[35,1296,1297,1300,1302,1304,1306,1309,1311,1313],{"class":37,"line":106},[35,1298,1299],{"class":82},"pandas_side[",[35,1301,367],{"class":45},[35,1303,489],{"class":82},[35,1305,119],{"class":78},[35,1307,1308],{"class":136}," int",[35,1310,1289],{"class":82},[35,1312,308],{"class":45},[35,1314,1315],{"class":82},"].count())\n",[35,1317,1318,1320,1322,1324,1326,1328,1330,1332],{"class":37,"line":113},[35,1319,1299],{"class":82},[35,1321,372],{"class":45},[35,1323,489],{"class":82},[35,1325,119],{"class":78},[35,1327,1308],{"class":136},[35,1329,1289],{"class":82},[35,1331,308],{"class":45},[35,1333,1334],{"class":82},"].isna().sum())\n",[35,1336,1337,1339,1341,1343,1346,1348,1351,1354,1356,1358],{"class":37,"line":172},[35,1338,527],{"class":136},[35,1340,436],{"class":82},[35,1342,367],{"class":45},[35,1344,1345],{"class":82},", pandas_side[",[35,1347,367],{"class":45},[35,1349,1350],{"class":82},"], ",[35,1352,1353],{"class":45},"\"| COUNTBLANK\"",[35,1355,1345],{"class":82},[35,1357,372],{"class":45},[35,1359,311],{"class":82},[10,1361,1362,1363,1366,1367,1370,1371,1373,1374,1376,1377,1380,1381,18],{},"With ",[32,1364,1365],{},"keep_default_na=False"," pandas no longer treats \"N\u002FA\" specially, and ",[32,1368,1369],{},"na_values=[\"\"]"," keeps empty cells as ",[32,1372,540],{}," — without it they would come back as empty strings. Now \"N\u002FA\" is a value, so ",[32,1375,1190],{}," gives Excel's 5 and the blank count 3. For arithmetic on such a column you then convert explicitly with ",[32,1378,1379],{},"pd.to_numeric(raw[\"Sales\"], errors=\"coerce\")",". Which reading is right depends on the question: for totals and averages the default is what you want; for auditing data entry (\"how many cells did people fill in?\") the raw reading is. More on placeholder handling in ",[14,1382,1384],{"href":1383},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Freplace-placeholder-text-with-nan-in-pandas\u002F","Replace Placeholder Text with NaN in pandas",[20,1386,1388],{"id":1387},"step-4-the-all-blank-group-0-or-nothing","Step 4 — The all-blank group: 0 or nothing?",[10,1390,1391],{},"Group sums are where pandas and Excel genuinely disagree. Summarise by region:",[25,1393,1395],{"className":69,"code":1394,"language":71,"meta":30,"style":30},"by_region = df.groupby(\"Region\")[\"Sales\"].agg([\"sum\", \"mean\", \"count\", \"size\"])\nby_region[\"sum (min_count=1)\"] = df.groupby(\"Region\")[\"Sales\"].sum(min_count=1)\nprint(by_region.to_string())\nprint(\"Excel AVERAGEIF for West:\", excel[\"AVERAGEIF West\"])\n",[32,1396,1397,1436,1469,1476],{"__ignoreMap":30},[35,1398,1399,1402,1404,1407,1409,1411,1413,1416,1419,1421,1424,1426,1429,1431,1434],{"class":37,"line":38},[35,1400,1401],{"class":82},"by_region ",[35,1403,119],{"class":78},[35,1405,1406],{"class":82}," df.groupby(",[35,1408,298],{"class":45},[35,1410,661],{"class":82},[35,1412,308],{"class":45},[35,1414,1415],{"class":82},"].agg([",[35,1417,1418],{"class":45},"\"sum\"",[35,1420,128],{"class":82},[35,1422,1423],{"class":45},"\"mean\"",[35,1425,128],{"class":82},[35,1427,1428],{"class":45},"\"count\"",[35,1430,128],{"class":82},[35,1432,1433],{"class":45},"\"size\"",[35,1435,311],{"class":82},[35,1437,1438,1441,1444,1446,1448,1450,1452,1454,1456,1459,1462,1464,1467],{"class":37,"line":92},[35,1439,1440],{"class":82},"by_region[",[35,1442,1443],{"class":45},"\"sum (min_count=1)\"",[35,1445,489],{"class":82},[35,1447,119],{"class":78},[35,1449,1406],{"class":82},[35,1451,298],{"class":45},[35,1453,661],{"class":82},[35,1455,308],{"class":45},[35,1457,1458],{"class":82},"].sum(",[35,1460,1461],{"class":445},"min_count",[35,1463,119],{"class":78},[35,1465,1466],{"class":136},"1",[35,1468,521],{"class":82},[35,1470,1471,1473],{"class":37,"line":106},[35,1472,527],{"class":136},[35,1474,1475],{"class":82},"(by_region.to_string())\n",[35,1477,1478,1480,1482,1485,1488,1491],{"class":37,"line":113},[35,1479,527],{"class":136},[35,1481,436],{"class":82},[35,1483,1484],{"class":45},"\"Excel AVERAGEIF for West:\"",[35,1486,1487],{"class":82},", excel[",[35,1489,1490],{"class":45},"\"AVERAGEIF West\"",[35,1492,311],{"class":82},[10,1494,1495,1496,1499,1500,1503,1504,1507,1508,1510,1511,1514,1515,1517,1518,1521,1522,1524,1525,1528,1529,1532,1533,1535,1536,18],{},"The West region has two reps and no figures. pandas reports its ",[32,1497,1498],{},"sum"," as ",[1501,1502,166],"strong",{}," — the sum of nothing is defined as zero — while its ",[32,1505,1506],{},"mean"," is ",[32,1509,540],{},". A report showing \"West: 0\" is wrong in a dangerous way: it says the region sold nothing, when in fact nobody knows. ",[32,1512,1513],{},"sum(min_count=1)"," fixes this: it requires at least one real value and otherwise returns ",[32,1516,540],{},", which shows as a blank in Excel. ",[32,1519,1520],{},"count"," (0) next to ",[32,1523,1206],{}," (2) is the honest picture: two reps, no figures. Excel's ",[32,1526,1527],{},"SUMIF"," behaves like pandas' default here and returns 0 for the West, so the same trap exists in formulas — adding a ",[32,1530,1531],{},"COUNTIF"," column beside every ",[32,1534,1527],{}," is the spreadsheet equivalent of ",[32,1537,1461],{},[808,1539,817,1544,817,1547,817,1550,817,1553,817,1561,817,1572,817,1577,817,1582,817,1586,817,1590,817,1594,817,1599,817,1606,817,1611,817,1615,817,1618,817,1621],{"viewBox":1540,"role":811,"ariaLabelledBy":1541,"xmlns":815,"style":816},"4 0 752 207",[1542,1543],"mv-allblank-t","mv-allblank-d",[819,1545,1546],{"id":1542},"Summing a group with no values",[823,1548,1549],{"id":1543},"For a region where every value is blank, pandas sum returns 0 and Excel SUMIF returns 0, which reads as a real zero. pandas sum with min_count 1 returns NaN, which shows as blank, and showing the count of values alongside makes the gap explicit.",[827,1551],{"x":829,"y":166,"width":830,"height":1552,"fill":832},"207",[827,1554],{"x":835,"y":836,"width":1555,"height":1556,"rx":1557,"fill":1558,"stroke":1559,"style":1560},"351.0","175.0","14","#fce9e9","var(--danger,#dc2626)","stroke-width:2px",[1562,1563,1565],"g",{"transform":1564},"translate(182.5,29.0) scale(1.083)",[1566,1567],"path",{"d":1568,"fill":1569,"stroke":1570,"style":1571},"M12 3.5l9.5 17h-19z M12 10v5 M12 17.8v.4","none","var(--accent-ink,#be185d)","stroke-width:1.66px;stroke-linecap:round;stroke-linejoin:round",[849,1573,1576],{"x":1574,"y":880,"style":1575},"195.5","font-size:14px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","default sum \u002F SUMIF",[37,1578],{"x1":1579,"y1":1580,"x2":1581,"y2":1580,"stroke":1559,"style":840},"36.0","91.0","355.0",[849,1583,1585],{"x":1574,"y":915,"style":1584},"font-size:12px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","West sum = 0",[849,1587,1589],{"x":1574,"y":1588,"style":1584},"137.0","reads as 'sold nothing'",[849,1591,1593],{"x":1574,"y":1592,"style":1584},"163.0","indistinguishable from a real zero",[827,1595],{"x":1596,"y":836,"width":1555,"height":1556,"rx":1557,"fill":1597,"stroke":1598,"style":1560},"389.0","#d9f4f1","var(--teal,#0f9488)",[1562,1600,1602],{"transform":1601},"translate(551.5,29.0) scale(1.083)",[1566,1603],{"d":1604,"fill":1569,"stroke":1605,"style":1571},"M5 12.5l4.5 4.5L19 7","var(--teal-ink,#0b6157)",[849,1607,1610],{"x":1608,"y":880,"style":1609},"564.5","font-size:14px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","sum(min_count=1) + count",[37,1612],{"x1":1613,"y1":1580,"x2":1614,"y2":1580,"stroke":1598,"style":840},"405.0","724.0",[849,1616,1617],{"x":1608,"y":915,"style":1584},"West sum = blank",[849,1619,1620],{"x":1608,"y":1588,"style":1584},"count = 0 of 2 reps",[849,1622,1623],{"x":1608,"y":1592,"style":1584},"gap is visible in the report",[20,1625,1627],{"id":1626},"step-5-blanks-versus-zeros-in-averages","Step 5 — Blanks versus zeros in averages",[10,1629,1630],{},"Whether a blank means \"zero\" or \"unknown\" changes the average more than anything else. Compare the two readings of the same column:",[25,1632,1634],{"className":69,"code":1633,"language":71,"meta":30,"style":30},"as_unknown = sales.mean()\nas_zero = sales.fillna(0).mean()\nprint(f\"blanks as unknown: {as_unknown:,.2f}  (over {sales.count()} values)\")\nprint(f\"blanks as zero:    {as_zero:,.2f}  (over {len(sales)} values)\")\n",[32,1635,1636,1646,1661,1697],{"__ignoreMap":30},[35,1637,1638,1641,1643],{"class":37,"line":38},[35,1639,1640],{"class":82},"as_unknown ",[35,1642,119],{"class":78},[35,1644,1645],{"class":82}," sales.mean()\n",[35,1647,1648,1651,1653,1656,1658],{"class":37,"line":92},[35,1649,1650],{"class":82},"as_zero ",[35,1652,119],{"class":78},[35,1654,1655],{"class":82}," sales.fillna(",[35,1657,166],{"class":136},[35,1659,1660],{"class":82},").mean()\n",[35,1662,1663,1665,1667,1669,1672,1674,1677,1680,1682,1685,1687,1690,1692,1695],{"class":37,"line":106},[35,1664,527],{"class":136},[35,1666,436],{"class":82},[35,1668,739],{"class":78},[35,1670,1671],{"class":45},"\"blanks as unknown: ",[35,1673,746],{"class":745},[35,1675,1676],{"class":82},"as_unknown",[35,1678,1679],{"class":78},":,.2f",[35,1681,755],{"class":745},[35,1683,1684],{"class":45},"  (over ",[35,1686,746],{"class":745},[35,1688,1689],{"class":82},"sales.count()",[35,1691,755],{"class":745},[35,1693,1694],{"class":45}," values)\"",[35,1696,521],{"class":82},[35,1698,1699,1701,1703,1705,1708,1710,1713,1715,1717,1719,1721,1723,1726,1728,1730],{"class":37,"line":113},[35,1700,527],{"class":136},[35,1702,436],{"class":82},[35,1704,739],{"class":78},[35,1706,1707],{"class":45},"\"blanks as zero:    ",[35,1709,746],{"class":745},[35,1711,1712],{"class":82},"as_zero",[35,1714,1679],{"class":78},[35,1716,755],{"class":745},[35,1718,1684],{"class":45},[35,1720,746],{"class":745},[35,1722,1114],{"class":136},[35,1724,1725],{"class":82},"(sales)",[35,1727,755],{"class":745},[35,1729,1694],{"class":45},[35,1731,521],{"class":82},[10,1733,1734,1735,1738,1739,1743],{},"Treating the four missing figures as unknown gives an average of 812.50; treating them as zero halves it to 406.25. Both can be correct — if a blank means \"no sales were made\", it ",[779,1736,1737],{},"is"," a zero and should be entered as one; if it means \"not reported yet\", it must stay blank. The worst outcome is mixing the two meanings in one column, which is why Chloé's real zero and Ben's blank should never be \"cleaned\" into the same thing. Decide what a blank means for each column, write it down — a data dictionary sheet is a good place, see ",[14,1740,1742],{"href":1741},"\u002Fautomating-reporting-workflows\u002Fbuilding-multi-sheet-excel-dashboards\u002Fadd-a-data-dictionary-sheet-to-an-excel-report\u002F","Add a Data Dictionary Sheet to an Excel Report"," — and fill or keep blanks accordingly before summarising.",[20,1745,1747],{"id":1746},"step-6-prove-pandas-and-excel-agree","Step 6 — Prove pandas and Excel agree",[10,1749,1750],{},"When a report is produced by pandas but checked by someone with a spreadsheet, a reconciliation at the end catches any difference in treatment:",[25,1752,1754],{"className":69,"code":1753,"language":71,"meta":30,"style":30},"mismatch = {k: (pandas_side[k], excel[k]) for k in pandas_side\n            if round(float(pandas_side[k]), 6) != round(float(excel[k]), 6)}\nprint(\"differences between pandas and Excel:\", mismatch or \"none\")\n",[32,1755,1756,1776,1813],{"__ignoreMap":30},[35,1757,1758,1761,1763,1766,1768,1771,1773],{"class":37,"line":38},[35,1759,1760],{"class":82},"mismatch ",[35,1762,119],{"class":78},[35,1764,1765],{"class":82}," {k: (pandas_side[k], excel[k]) ",[35,1767,317],{"class":78},[35,1769,1770],{"class":82}," k ",[35,1772,323],{"class":78},[35,1774,1775],{"class":82}," pandas_side\n",[35,1777,1778,1781,1784,1786,1789,1792,1794,1796,1799,1801,1803,1805,1808,1810],{"class":37,"line":92},[35,1779,1780],{"class":78},"            if",[35,1782,1783],{"class":136}," round",[35,1785,436],{"class":82},[35,1787,1788],{"class":136},"float",[35,1790,1791],{"class":82},"(pandas_side[k]), ",[35,1793,474],{"class":136},[35,1795,706],{"class":82},[35,1797,1798],{"class":78},"!=",[35,1800,1783],{"class":136},[35,1802,436],{"class":82},[35,1804,1788],{"class":136},[35,1806,1807],{"class":82},"(excel[k]), ",[35,1809,474],{"class":136},[35,1811,1812],{"class":82},")}\n",[35,1814,1815,1817,1819,1822,1825,1828,1831],{"class":37,"line":106},[35,1816,527],{"class":136},[35,1818,436],{"class":82},[35,1820,1821],{"class":45},"\"differences between pandas and Excel:\"",[35,1823,1824],{"class":82},", mismatch ",[35,1826,1827],{"class":78},"or",[35,1829,1830],{"class":45}," \"none\"",[35,1832,521],{"class":82},[10,1834,1835,1836,1196,1838,1840,1841,1843,1844,1848],{},"With the same definitions on both sides — numeric figures from the default read, ",[32,1837,789],{},[32,1839,793],{}," from the raw read — every figure matches. Without the second read, this check is exactly what would have flagged the ",[32,1842,789],{}," difference. Running a check like this against the formulas in the delivered workbook — recalculated with LibreOffice as above, or with a formula engine as in ",[14,1845,1847],{"href":1846},"\u002Fgetting-started-with-python-excel-automation\u002Fworking-with-excel-formulas-in-python\u002Frecalculate-excel-formulas-without-excel-in-python\u002F","Recalculate Excel Formulas Without Excel in Python"," — turns \"why don't my numbers match yours?\" into a test that fails before anyone asks.",[808,1850,817,1855,817,1858,817,1861,817,1864,817,1869,817,1877,817,1883,817,1887,817,1892,817,1897,817,1900,817,1907,817,1912,817,1915,817,1918,817,1921,817,1924,817,1930,817,1936,817,1940,817,1944,817,1948,817,1951,817,1956,817,1962,817,1967],{"viewBox":1851,"role":811,"ariaLabelledBy":1852,"xmlns":815,"style":816},"4 0 752 142",[1853,1854],"mv-habit-t","mv-habit-d",[819,1856,1857],{"id":1853},"A summary that both tools agree on",[823,1859,1860],{"id":1854},"Decide what a blank means for each column, convert placeholder text to missing values, summarise with min_count and show counts alongside sums and averages, then reconcile with the spreadsheet's own formulas.",[827,1862],{"x":829,"y":166,"width":830,"height":1863,"fill":832},"142",[827,1865],{"x":835,"y":836,"width":1866,"height":1867,"rx":1868,"fill":845,"stroke":846,"style":1560},"153.0","110.5","12",[1562,1870,1872],{"transform":1871},"translate(84.5,37.8) scale(1.000)",[1566,1873],{"d":1874,"fill":1569,"stroke":1875,"style":1876},"M11 4a7 7 0 1 0 .01 0z M16 16l5 5","var(--brand-strong,#4338ca)","stroke-width:1.80px;stroke-linecap:round;stroke-linejoin:round",[849,1878,1882],{"x":1879,"y":1880,"style":1881},"96.5","84.2","font-size:13.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Define blanks",[849,1884,1886],{"x":1879,"y":1885,"style":888},"103.8","unknown or zero?",[37,1888],{"x1":1889,"y1":1890,"x2":1891,"y2":1890,"stroke":846,"style":1560},"178.0","71.2","200.0",[1893,1894],"polygon",{"points":1895,"fill":1896},"204.0,71.2 195.0,76.2 195.0,66.2","#5b5cf0",[827,1898],{"x":1899,"y":836,"width":1866,"height":1867,"rx":1868,"fill":876,"stroke":839,"style":1560},"209.0",[1562,1901,1903],{"transform":1902},"translate(273.5,37.8) scale(1.000)",[1566,1904],{"d":1905,"fill":1569,"stroke":1906,"style":1876},"M20 11a8 8 0 1 0-2.3 5.7 M20 4v7h-7","var(--text,#172033)",[849,1908,1911],{"x":1909,"y":1880,"style":1910},"285.5","font-size:13.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","Convert text",[849,1913,1914],{"x":1909,"y":1885,"style":888},"'N\u002FA' → NaN",[37,1916],{"x1":1917,"y1":1890,"x2":1596,"y2":1890,"stroke":846,"style":1560},"367.0",[1893,1919],{"points":1920,"fill":1896},"393.0,71.2 384.0,76.2 384.0,66.2",[827,1922],{"x":1923,"y":836,"width":1866,"height":1867,"rx":1868,"fill":1597,"stroke":1598,"style":1560},"398.0",[1562,1925,1927],{"transform":1926},"translate(462.5,29.0) scale(1.000)",[1566,1928],{"d":1929,"fill":1569,"stroke":1605,"style":1876},"M5 9h14 M4 15h14 M10 4l-2 16 M16 4l-2 16",[849,1931,1935],{"x":1932,"y":1933,"style":1934},"474.5","75.5","font-size:13.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","Summarise",[849,1937,1939],{"x":1932,"y":1938,"style":888},"95.0","min_count=1, counts",[849,1941,1943],{"x":1932,"y":1942,"style":888},"112.5","shown",[37,1945],{"x1":1946,"y1":1890,"x2":1947,"y2":1890,"stroke":846,"style":1560},"556.0","578.0",[1893,1949],{"points":1950,"fill":1896},"582.0,71.2 573.0,76.2 573.0,66.2",[827,1952],{"x":1953,"y":836,"width":1866,"height":1867,"rx":1868,"fill":1954,"stroke":1955,"style":1560},"587.0","#fdefd8","var(--gold,#b4740a)",[1562,1957,1959],{"transform":1958},"translate(651.5,37.8) scale(1.000)",[1566,1960],{"d":1604,"fill":1569,"stroke":1961,"style":1876},"var(--gold-ink,#7a4e06)",[849,1963,1966],{"x":1964,"y":1880,"style":1965},"663.5","font-size:13.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","Reconcile",[849,1968,1969],{"x":1964,"y":1885,"style":888},"pandas vs formulas",[20,1971,1973],{"id":1972},"common-pitfalls","Common pitfalls",[1975,1976,1977,1993,2002,2008,2018],"ul",{},[1978,1979,1980,1983,1984,1986,1987,1989,1990,1992],"li",{},[1501,1981,1982],{},"Forgetting pandas' default placeholders."," ",[32,1985,1225],{}," turns \"N\u002FA\" and similar text into ",[32,1988,540],{},"; use ",[32,1991,1365],{}," when that matters.",[1978,1994,1995,1998,1999,2001],{},[1501,1996,1997],{},"All-blank groups summing to 0."," Use ",[32,2000,1513],{}," and show the count.",[1978,2003,2004,2007],{},[1501,2005,2006],{},"Filling blanks with 0 to \"fix\" averages."," Only if a blank really means zero.",[1978,2009,2010,2017],{},[1501,2011,2012,2013,1196,2015,18],{},"Confusing ",[32,2014,1520],{},[32,2016,1206],{}," One counts values, the other rows.",[1978,2019,2020,2023],{},[1501,2021,2022],{},"No reconciliation."," Compare pandas results with the workbook's formulas.",[20,2025,2027],{"id":2026},"conclusion","Conclusion",[10,2029,2030,2031,128,2033,1196,2035,2037,2038,2040,2041,2043,2044,128,2046,1196,2048,2050,2051,1196,2053,2055,2056,2059],{},"Excel's ",[32,2032,772],{},[32,2034,776],{},[32,2036,785],{}," ignore blanks and text but include zeros; ",[32,2039,789],{}," also counts text. pandas reads common placeholders such as \"N\u002FA\" as ",[32,2042,540],{}," by default, after which ",[32,2045,1498],{},[32,2047,1506],{},[32,2049,1520],{}," skip missing values, while ",[32,2052,1206],{},[32,2054,1114],{}," count rows. The traps are an all-blank group summing to 0 — use ",[32,2057,2058],{},"min_count=1"," and report counts — and treating blanks as zeros, which can halve an average. Decide what a blank means per column, and reconcile pandas figures with spreadsheet formulas before publishing.",[20,2061,2063],{"id":2062},"frequently-asked-questions","Frequently asked questions",[10,2065,2066,2069],{},[1501,2067,2068],{},"Does Excel's AVERAGE ignore blank cells?","\nYes. AVERAGE divides by the number of numeric cells, so blanks and text are left out, but zeros are included and lower the average.",[10,2071,2072,2075],{},[1501,2073,2074],{},"Why does pandas sum return 0 for a group with no values?","\nThe sum of no values is defined as 0. Pass min_count=1 to sum so that a group needs at least one real value, otherwise it returns NaN.",[10,2077,2078,2081],{},[1501,2079,2080],{},"What is the difference between count and size in pandas?","\ncount returns the number of non-missing values; size returns the number of rows, including those with missing values.",[10,2083,2084,2087],{},[1501,2085,2086],{},"Should I replace blanks with zero before averaging?","\nOnly if a blank really means zero. If it means unknown, keep it blank, because filling with zero lowers the average and hides the gap.",[20,2089,2091],{"id":2090},"related","Related",[1975,2093,2094,2100,2107,2114],{},[1978,2095,2096,2097,2099],{},"Up one level: ",[14,2098,17],{"href":16}," — finding and treating gaps.",[1978,2101,2102,2106],{},[14,2103,2105],{"href":2104},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ftell-blank-cells-empty-strings-and-zeros-apart-in-excel-data\u002F","Tell Blank Cells, Empty Strings and Zeros Apart in Excel Data"," — what a blank really is.",[1978,2108,2109,2113],{},[14,2110,2112],{"href":2111},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fcountif-and-countifs-equivalent-in-pandas\u002F","COUNTIF and COUNTIFS Equivalent in pandas"," — conditional counts.",[1978,2115,2116,2120],{},[14,2117,2119],{"href":2118},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Faverageifs-maxifs-minifs-in-pandas\u002F","AVERAGEIFS, MAXIFS and MINIFS Equivalents in pandas"," — conditional averages.",[2122,2123,2124],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":30,"searchDepth":92,"depth":92,"links":2126},[2127,2128,2129,2130,2131,2132,2133,2134,2135,2136,2137],{"id":22,"depth":92,"text":23},{"id":62,"depth":92,"text":63},{"id":544,"depth":92,"text":545},{"id":981,"depth":92,"text":982},{"id":1387,"depth":92,"text":1388},{"id":1626,"depth":92,"text":1627},{"id":1746,"depth":92,"text":1747},{"id":1972,"depth":92,"text":1973},{"id":2026,"depth":92,"text":2027},{"id":2062,"depth":92,"text":2063},{"id":2090,"depth":92,"text":2091},"2026-10-02","Blank cells quietly change summary figures, and pandas and Excel do not always agree: what SUM, AVERAGE, COUNT and COUNTA do with blanks, text placeholders and zeros, how pandas sum, mean, count and size compare, why an all-blank group sums to 0, min_count, averages over blanks versus zeros, and a reconciliation that proves both tools give the same answers.","md",[2142,2144,2146,2148],{"q":2068,"a":2143},"Yes. AVERAGE divides by the number of numeric cells, so blanks and text are left out, but zeros are included and lower the average.",{"q":2074,"a":2145},"The sum of no values is defined as 0. Pass min_count=1 to sum so that a group needs at least one real value, otherwise it returns NaN.",{"q":2080,"a":2147},"count returns the number of non-missing values; size returns the number of rows, including those with missing values.",{"q":2086,"a":2149},"Only if a blank really means zero. If it means unknown, keep it blank, because filling with zero lowers the average and hides the gap.","how missing values change sums averages and counts",null,{"breadcrumb":2153},[2154,2157,2160],{"name":2155,"item":2156},"Home","\u002F",{"name":2158,"item":2159},"Advanced Data Transformation and Cleaning for Excel Automation","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":2161,"item":16},"Handling Missing Data in Excel Reports with Pandas","\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts",{"title":2164,"description":2165},"Missing Values in Sums, Averages and Counts (pandas vs Excel)","Blanks versus zeros in summaries: Excel SUM, AVERAGE, COUNT, COUNTA and COUNTBLANK against pandas sum, mean, count, size and min_count, all-NaN groups, placeholder text and a reconciliation check.","how-missing-values-change-sums-averages-and-counts","advanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fhow-missing-values-change-sums-averages-and-counts\u002Findex",[2169,2170],"missing-data","pivots-and-summaries","how-to","BVuYgt4bhvcHGDJCBYdP0gMg2P9UsdooXPi_uQoI0nU",[2174,2178],{"title":2175,"path":2176,"stem":2177,"children":-1},"Forward-Fill Blank Cells in Grouped Excel Reports with pandas","\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fforward-fill-blank-cells-in-grouped-excel-reports","advanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fforward-fill-blank-cells-in-grouped-excel-reports\u002Findex",{"title":2179,"path":2180,"stem":2181,"children":-1},"Impute Missing Values by Group with pandas","\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fimpute-missing-values-by-group-with-pandas","advanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fimpute-missing-values-by-group-with-pandas\u002Findex",[2183,2185],{"path":2184,"title":2161},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports",{"path":2186,"title":2158},"\u002Fadvanced-data-transformation-and-cleaning",[],[2189,2192,2195,2198,2201,2204],{"path":2190,"title":2191,"kind":2171},"\u002Fadvanced-data-transformation-and-cleaning\u002Fapplying-conditional-formatting-with-openpyxl\u002Fhighlight-blank-cells-with-openpyxl","Highlight Blank Cells in Excel with openpyxl",{"path":2193,"title":2194,"kind":2171},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fcoalesce-several-excel-columns-into-one-with-pandas","Coalesce Several Excel Columns into One with pandas",{"path":2196,"title":2197,"kind":2171},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fiferror-and-isna-equivalents-in-pandas","IFERROR, IFNA, ISNA and ISNUMBER Equivalents in pandas",{"path":2199,"title":2200,"kind":2171},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fdrop-rows-with-missing-required-fields-in-pandas","Drop Rows with Missing Required Fields in pandas",{"path":2202,"title":2203,"kind":2171},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffill-gaps-in-per-item-time-series-with-a-staleness-limit","Fill Gaps in Per-Item Time Series with a Staleness Limit",{"path":2205,"title":2206,"kind":2171},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffill-missing-values-in-excel-with-pandas-fillna","Fill Missing Values in Excel with Pandas fillna",1790972096003]