[{"data":1,"prerenderedAt":2501},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fcountif-and-countifs-equivalent-in-pandas":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fcountif-and-countifs-equivalent-in-pandas":2493},{"id":4,"title":5,"body":6,"dateModified":2465,"datePublished":2465,"description":2466,"extension":2467,"faq":2468,"meta":2477,"navigation":198,"path":2486,"seo":2487,"slug":2489,"stem":2490,"type":2491,"__hash__":2492},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fcountif-and-countifs-equivalent-in-pandas\u002Findex.md","COUNTIF and COUNTIFS Equivalent in pandas",{"type":7,"value":8,"toc":2449},"minimark",[9,23,138,143,170,516,520,628,644,708,711,715,718,795,811,827,835,839,921,935,1010,1018,1103,1110,1114,1117,1218,1221,1225,1339,1348,1352,1355,1494,1501,1516,1520,1523,1713,1724,1728,1845,1849,1852,2071,2078,2085,2089,2092,2192,2328,2338,2342,2361,2365,2372,2385,2397,2407,2411,2445],[10,11,12,16,17,22],"p",{},[13,14,15],"code",{},"COUNTIF"," answers \"how many rows match\", which in pandas is the same boolean mask as a conditional\nsum with a different final step. What makes counting worth its own guide is the family of related\nquestions around it — how many distinct values, how many blanks, what proportion — several of which\nneed an awkward array formula in Excel and one method call here. This guide is part of\n",[18,19,21],"a",{"href":20},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002F","Excel Formula Equivalents in pandas",".",[24,25,33,34,33,38,33,42,33,49,33,59,33,66,33,73,33,78,33,82,33,86,33,91,33,96,33,100,33,103,33,106,33,109,33,114,33,119,33,123,33,126,33,129,33,132],"svg",{"viewBox":26,"role":27,"ariaLabelledBy":28,"xmlns":31,"style":32},"0 0 760 209","img",[29,30],"cif-family-t","cif-family-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:760px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[35,36,37],"title",{"id":29},"The counting questions, and where each lives",[39,40,41],"desc",{"id":30},"Excel spreads counting across COUNT, COUNTA, COUNTBLANK, COUNTIF and an array formula for distinct values, while pandas answers each with a single method on the column.",[43,44],"rect",{"x":45,"y":45,"width":46,"height":47,"fill":48},"0","760","209","#ffffff",[43,50],{"x":51,"y":52,"width":53,"height":54,"rx":55,"fill":56,"stroke":57,"style":58},"20.0","26","226.7","141","14","#d9f4f1","var(--teal,#0f9488)","stroke-width:2px",[60,61,65],"text",{"x":62,"y":63,"style":64},"133.35","52","font-size:13px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","count rows matching",[67,68],"line",{"x1":69,"y1":70,"x2":71,"y2":70,"stroke":57,"style":72},"36.0","62","230.7","stroke-width:1px",[60,74,77],{"x":62,"y":75,"style":76},"84","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","mask.sum()",[60,79,81],{"x":62,"y":80,"style":76},"107","mask.mean() for a share",[60,83,85],{"x":62,"y":84,"style":76},"130","one expression",[43,87],{"x":88,"y":52,"width":53,"height":54,"rx":55,"fill":89,"stroke":90,"style":58},"266.7","#f0f4ff","var(--brand,#5b5cf0)",[60,92,95],{"x":93,"y":63,"style":94},"380.04999999999995","font-size:13px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","count values present",[67,97],{"x1":98,"y1":70,"x2":99,"y2":70,"stroke":90,"style":72},"282.7","477.4",[60,101,102],{"x":93,"y":75,"style":76},"col.count()",[60,104,105],{"x":93,"y":80,"style":76},"col.isna().sum()",[60,107,108],{"x":93,"y":84,"style":76},"empty strings are not NaN",[43,110],{"x":111,"y":52,"width":53,"height":54,"rx":55,"fill":112,"stroke":113,"style":58},"513.4","#fdefd8","var(--gold,#b4740a)",[60,115,118],{"x":116,"y":63,"style":117},"626.75","font-size:13px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","count distinct",[67,120],{"x1":121,"y1":70,"x2":122,"y2":70,"stroke":113,"style":72},"529.4","724.0999999999999",[60,124,125],{"x":116,"y":75,"style":76},"col.nunique()",[60,127,128],{"x":116,"y":80,"style":76},"col.value_counts()",[60,130,131],{"x":116,"y":84,"style":76},"an array formula in Excel",[60,133,137],{"x":134,"y":135,"style":136},"380.0","195","font-size:12.5px;font-weight:400;fill:var(--muted,#5b6780);text-anchor:middle","three questions, three methods, no array formulas",[139,140,142],"h2",{"id":141},"prerequisites","Prerequisites",[144,145,150],"pre",{"className":146,"code":147,"language":148,"meta":149,"style":149},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash","",[13,151,152],{"__ignoreMap":149},[153,154,156,160,164,167],"span",{"class":67,"line":155},1,[153,157,159],{"class":158},"sMTad","pip",[153,161,163],{"class":162},"srMev"," install",[153,165,166],{"class":162}," pandas",[153,168,169],{"class":162}," openpyxl\n",[144,171,175],{"className":172,"code":173,"language":174,"meta":149,"style":149},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\nsales = pd.DataFrame({\n    \"Region\": [\"North\", \"South\", \"North\", \"West\", \"South\", \"North\", \"West\"],\n    \"Rep\": [\"Ana\", \"Ben\", \"Cara\", \"Dev\", \"Eve\", \"Ana\", \"Dev\"],\n    \"Product\": [\"Widget\", \"Gadget\", \"Widget\", \"Widget\", \"Gadget\", \"Gadget\", \"Widget\"],\n    \"Revenue\": [12400.0, 9800.5, 15320.25, 7010.0, 4300.75, 6120.0, 11450.5],\n    \"Units\": [124, 98, 153, 70, 43, 61, 114],\n    \"Ordered\": pd.to_datetime([\n        \"2026-01-04\", \"2026-01-19\", \"2026-02-02\", \"2026-02-27\",\n        \"2026-03-08\", \"2026-03-15\", \"2026-03-30\",\n    ]),\n})\nsales.to_excel(\"sales.xlsx\", sheet_name=\"Detail\", index=False)\n","python",[13,176,177,193,200,212,254,295,333,377,420,429,453,471,477,483],{"__ignoreMap":149},[153,178,179,183,187,190],{"class":67,"line":155},[153,180,182],{"class":181},"s-kum","import",[153,184,186],{"class":185},"skGVy"," pandas ",[153,188,189],{"class":181},"as",[153,191,192],{"class":185}," pd\n",[153,194,196],{"class":67,"line":195},2,[153,197,199],{"emptyLinePlaceholder":198},true,"\n",[153,201,203,206,209],{"class":67,"line":202},3,[153,204,205],{"class":185},"sales ",[153,207,208],{"class":181},"=",[153,210,211],{"class":185}," pd.DataFrame({\n",[153,213,215,218,221,224,227,230,232,234,236,239,241,243,245,247,249,251],{"class":67,"line":214},4,[153,216,217],{"class":162},"    \"Region\"",[153,219,220],{"class":185},": [",[153,222,223],{"class":162},"\"North\"",[153,225,226],{"class":185},", ",[153,228,229],{"class":162},"\"South\"",[153,231,226],{"class":185},[153,233,223],{"class":162},[153,235,226],{"class":185},[153,237,238],{"class":162},"\"West\"",[153,240,226],{"class":185},[153,242,229],{"class":162},[153,244,226],{"class":185},[153,246,223],{"class":162},[153,248,226],{"class":185},[153,250,238],{"class":162},[153,252,253],{"class":185},"],\n",[153,255,257,260,262,265,267,270,272,275,277,280,282,285,287,289,291,293],{"class":67,"line":256},5,[153,258,259],{"class":162},"    \"Rep\"",[153,261,220],{"class":185},[153,263,264],{"class":162},"\"Ana\"",[153,266,226],{"class":185},[153,268,269],{"class":162},"\"Ben\"",[153,271,226],{"class":185},[153,273,274],{"class":162},"\"Cara\"",[153,276,226],{"class":185},[153,278,279],{"class":162},"\"Dev\"",[153,281,226],{"class":185},[153,283,284],{"class":162},"\"Eve\"",[153,286,226],{"class":185},[153,288,264],{"class":162},[153,290,226],{"class":185},[153,292,279],{"class":162},[153,294,253],{"class":185},[153,296,298,301,303,306,308,311,313,315,317,319,321,323,325,327,329,331],{"class":67,"line":297},6,[153,299,300],{"class":162},"    \"Product\"",[153,302,220],{"class":185},[153,304,305],{"class":162},"\"Widget\"",[153,307,226],{"class":185},[153,309,310],{"class":162},"\"Gadget\"",[153,312,226],{"class":185},[153,314,305],{"class":162},[153,316,226],{"class":185},[153,318,305],{"class":162},[153,320,226],{"class":185},[153,322,310],{"class":162},[153,324,226],{"class":185},[153,326,310],{"class":162},[153,328,226],{"class":185},[153,330,305],{"class":162},[153,332,253],{"class":185},[153,334,336,339,341,345,347,350,352,355,357,360,362,365,367,370,372,375],{"class":67,"line":335},7,[153,337,338],{"class":162},"    \"Revenue\"",[153,340,220],{"class":185},[153,342,344],{"class":343},"sP0c6","12400.0",[153,346,226],{"class":185},[153,348,349],{"class":343},"9800.5",[153,351,226],{"class":185},[153,353,354],{"class":343},"15320.25",[153,356,226],{"class":185},[153,358,359],{"class":343},"7010.0",[153,361,226],{"class":185},[153,363,364],{"class":343},"4300.75",[153,366,226],{"class":185},[153,368,369],{"class":343},"6120.0",[153,371,226],{"class":185},[153,373,374],{"class":343},"11450.5",[153,376,253],{"class":185},[153,378,380,383,385,388,390,393,395,398,400,403,405,408,410,413,415,418],{"class":67,"line":379},8,[153,381,382],{"class":162},"    \"Units\"",[153,384,220],{"class":185},[153,386,387],{"class":343},"124",[153,389,226],{"class":185},[153,391,392],{"class":343},"98",[153,394,226],{"class":185},[153,396,397],{"class":343},"153",[153,399,226],{"class":185},[153,401,402],{"class":343},"70",[153,404,226],{"class":185},[153,406,407],{"class":343},"43",[153,409,226],{"class":185},[153,411,412],{"class":343},"61",[153,414,226],{"class":185},[153,416,417],{"class":343},"114",[153,419,253],{"class":185},[153,421,423,426],{"class":67,"line":422},9,[153,424,425],{"class":162},"    \"Ordered\"",[153,427,428],{"class":185},": pd.to_datetime([\n",[153,430,432,435,437,440,442,445,447,450],{"class":67,"line":431},10,[153,433,434],{"class":162},"        \"2026-01-04\"",[153,436,226],{"class":185},[153,438,439],{"class":162},"\"2026-01-19\"",[153,441,226],{"class":185},[153,443,444],{"class":162},"\"2026-02-02\"",[153,446,226],{"class":185},[153,448,449],{"class":162},"\"2026-02-27\"",[153,451,452],{"class":185},",\n",[153,454,456,459,461,464,466,469],{"class":67,"line":455},11,[153,457,458],{"class":162},"        \"2026-03-08\"",[153,460,226],{"class":185},[153,462,463],{"class":162},"\"2026-03-15\"",[153,465,226],{"class":185},[153,467,468],{"class":162},"\"2026-03-30\"",[153,470,452],{"class":185},[153,472,474],{"class":67,"line":473},12,[153,475,476],{"class":185},"    ]),\n",[153,478,480],{"class":67,"line":479},13,[153,481,482],{"class":185},"})\n",[153,484,486,489,492,494,498,500,503,505,508,510,513],{"class":67,"line":485},14,[153,487,488],{"class":185},"sales.to_excel(",[153,490,491],{"class":162},"\"sales.xlsx\"",[153,493,226],{"class":185},[153,495,497],{"class":496},"sa561","sheet_name",[153,499,208],{"class":181},[153,501,502],{"class":162},"\"Detail\"",[153,504,226],{"class":185},[153,506,507],{"class":496},"index",[153,509,208],{"class":181},[153,511,512],{"class":343},"False",[153,514,515],{"class":185},")\n",[139,517,519],{"id":518},"countif-and-countifs","COUNTIF and COUNTIFS",[144,521,523],{"className":172,"code":522,"language":174,"meta":149,"style":149},"# =COUNTIF(A:A, \"North\")\nprint((sales[\"Region\"] == \"North\").sum())\n\n# =COUNTIFS(A:A, \"North\", C:C, \"Widget\")\nprint(((sales[\"Region\"] == \"North\") & (sales[\"Product\"] == \"Widget\")).sum())\n\n# =COUNTIF(D:D, \">10000\")\nprint((sales[\"Revenue\"] > 10000).sum())\n",[13,524,525,531,554,558,563,600,604,609],{"__ignoreMap":149},[153,526,527],{"class":67,"line":155},[153,528,530],{"class":529},"s-wDw","# =COUNTIF(A:A, \"North\")\n",[153,532,533,536,539,542,545,548,551],{"class":67,"line":195},[153,534,535],{"class":343},"print",[153,537,538],{"class":185},"((sales[",[153,540,541],{"class":162},"\"Region\"",[153,543,544],{"class":185},"] ",[153,546,547],{"class":181},"==",[153,549,550],{"class":162}," \"North\"",[153,552,553],{"class":185},").sum())\n",[153,555,556],{"class":67,"line":202},[153,557,199],{"emptyLinePlaceholder":198},[153,559,560],{"class":67,"line":214},[153,561,562],{"class":529},"# =COUNTIFS(A:A, \"North\", C:C, \"Widget\")\n",[153,564,565,567,570,572,574,576,578,581,584,587,590,592,594,597],{"class":67,"line":256},[153,566,535],{"class":343},[153,568,569],{"class":185},"(((sales[",[153,571,541],{"class":162},[153,573,544],{"class":185},[153,575,547],{"class":181},[153,577,550],{"class":162},[153,579,580],{"class":185},") ",[153,582,583],{"class":181},"&",[153,585,586],{"class":185}," (sales[",[153,588,589],{"class":162},"\"Product\"",[153,591,544],{"class":185},[153,593,547],{"class":181},[153,595,596],{"class":162}," \"Widget\"",[153,598,599],{"class":185},")).sum())\n",[153,601,602],{"class":67,"line":297},[153,603,199],{"emptyLinePlaceholder":198},[153,605,606],{"class":67,"line":335},[153,607,608],{"class":529},"# =COUNTIF(D:D, \">10000\")\n",[153,610,611,613,615,618,620,623,626],{"class":67,"line":379},[153,612,535],{"class":343},[153,614,538],{"class":185},[153,616,617],{"class":162},"\"Revenue\"",[153,619,544],{"class":185},[153,621,622],{"class":181},">",[153,624,625],{"class":343}," 10000",[153,627,553],{"class":185},[10,629,630,631,634,635,638,639,643],{},"The trick is that a boolean Series sums as ones and zeros, so ",[13,632,633],{},".sum()"," on a mask is a count. That\nalso means ",[13,636,637],{},".mean()"," on the same mask gives the ",[640,641,642],"em",{},"proportion"," matching, which Excel expresses as a\nCOUNTIF divided by a COUNTA:",[144,645,647],{"className":172,"code":646,"language":174,"meta":149,"style":149},"mask = sales[\"Revenue\"] > 10000\nprint(f\"{mask.sum()} orders, {mask.mean():.1%} of the total\")\n",[13,648,649,668],{"__ignoreMap":149},[153,650,651,654,656,659,661,663,665],{"class":67,"line":155},[153,652,653],{"class":185},"mask ",[153,655,208],{"class":181},[153,657,658],{"class":185}," sales[",[153,660,617],{"class":162},[153,662,544],{"class":185},[153,664,622],{"class":181},[153,666,667],{"class":343}," 10000\n",[153,669,670,672,675,678,681,685,687,690,693,695,698,701,703,706],{"class":67,"line":195},[153,671,535],{"class":343},[153,673,674],{"class":185},"(",[153,676,677],{"class":181},"f",[153,679,680],{"class":162},"\"",[153,682,684],{"class":683},"sSjpA","{",[153,686,77],{"class":185},[153,688,689],{"class":683},"}",[153,691,692],{"class":162}," orders, ",[153,694,684],{"class":683},[153,696,697],{"class":185},"mask.mean()",[153,699,700],{"class":181},":.1%",[153,702,689],{"class":683},[153,704,705],{"class":162}," of the total\"",[153,707,515],{"class":185},[10,709,710],{},"Getting a count and a share from the same object, without writing the condition twice, is the sort of\nsmall ergonomic gain that adds up across a report.",[139,712,714],{"id":713},"counting-values-blanks-and-distinct-entries","Counting values, blanks and distinct entries",[10,716,717],{},"Excel splits these across COUNT, COUNTA, COUNTBLANK and a SUMPRODUCT array formula. pandas has one\nmethod for each and they read the way they sound.",[144,719,721],{"className":172,"code":720,"language":174,"meta":149,"style":149},"print(sales[\"Revenue\"].count())        # =COUNT  — non-null numeric values\nprint(sales[\"Rep\"].notna().sum())      # =COUNTA — non-blank entries\nprint(sales[\"Rep\"].isna().sum())       # =COUNTBLANK\nprint(sales[\"Region\"].nunique())       # distinct regions\nprint(sales[\"Region\"].value_counts())  # the breakdown, sorted\n",[13,722,723,738,753,767,781],{"__ignoreMap":149},[153,724,725,727,730,732,735],{"class":67,"line":155},[153,726,535],{"class":343},[153,728,729],{"class":185},"(sales[",[153,731,617],{"class":162},[153,733,734],{"class":185},"].count())        ",[153,736,737],{"class":529},"# =COUNT  — non-null numeric values\n",[153,739,740,742,744,747,750],{"class":67,"line":195},[153,741,535],{"class":343},[153,743,729],{"class":185},[153,745,746],{"class":162},"\"Rep\"",[153,748,749],{"class":185},"].notna().sum())      ",[153,751,752],{"class":529},"# =COUNTA — non-blank entries\n",[153,754,755,757,759,761,764],{"class":67,"line":202},[153,756,535],{"class":343},[153,758,729],{"class":185},[153,760,746],{"class":162},[153,762,763],{"class":185},"].isna().sum())       ",[153,765,766],{"class":529},"# =COUNTBLANK\n",[153,768,769,771,773,775,778],{"class":67,"line":214},[153,770,535],{"class":343},[153,772,729],{"class":185},[153,774,541],{"class":162},[153,776,777],{"class":185},"].nunique())       ",[153,779,780],{"class":529},"# distinct regions\n",[153,782,783,785,787,789,792],{"class":67,"line":256},[153,784,535],{"class":343},[153,786,729],{"class":185},[153,788,541],{"class":162},[153,790,791],{"class":185},"].value_counts())  ",[153,793,794],{"class":529},"# the breakdown, sorted\n",[10,796,797,800,801,226,804,452,807,810],{},[13,798,799],{},"value_counts()"," is the one worth reaching for first when investigating a new file. It answers \"what\nis actually in this column\" in a single line, and it is how spelling variants — ",[13,802,803],{},"North",[13,805,806],{},"north",[13,808,809],{},"North "," — get discovered before they distort a total.",[144,812,814],{"className":172,"code":813,"language":174,"meta":149,"style":149},"print(sales[\"Region\"].str.strip().str.casefold().value_counts())\n",[13,815,816],{"__ignoreMap":149},[153,817,818,820,822,824],{"class":67,"line":155},[153,819,535],{"class":343},[153,821,729],{"class":185},[153,823,541],{"class":162},[153,825,826],{"class":185},"].str.strip().str.casefold().value_counts())\n",[10,828,829,830,834],{},"If the two counts differ, the column needs the cleanup described in\n",[18,831,833],{"href":832},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fstrip-whitespace-and-normalise-text-columns-with-pandas\u002F","Strip Whitespace and Normalise Text Columns with Pandas","\nbefore any conditional count means anything.",[139,836,838],{"id":837},"counts-per-group","Counts per group",[24,840,33,845,33,848,33,851,33,854,33,859,33,866,33,871,33,876,33,881,33,886,33,889,33,892,33,895,33,899,33,902,33,905,33,910,33,913,33,917],{"viewBox":841,"role":27,"ariaLabelledBy":842,"xmlns":31,"style":32},"0 0 760 232",[843,844],"cif-group-t","cif-group-d",[35,846,847],{"id":843},"Counting within groups",[39,849,850],{"id":844},"Grouping by a key and applying size distributes each group's row count back to every member row, which is what a COUNTIF referencing the current row's key does one row at a time.",[43,852],{"x":45,"y":45,"width":46,"height":853,"fill":48},"232",[60,855,858],{"x":134,"y":856,"style":857},"32","font-size:13px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","per-group counts",[43,860],{"x":861,"y":862,"width":863,"height":864,"rx":865,"fill":89,"stroke":90,"style":58},"24.0","74","208.0","96","12",[60,867,870],{"x":868,"y":417,"style":869},"128.0","font-size:14px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","groupby('Region')",[60,872,875],{"x":868,"y":873,"style":874},"136","font-size:11.5px;font-weight:400;fill:var(--muted,#5b6780);text-anchor:middle","one group per key",[67,877],{"x1":878,"y1":879,"x2":880,"y2":879,"stroke":90,"style":58},"237.0","122.0","269.0",[882,883],"polygon",{"points":884,"fill":885},"269.0,122.0 260.0,117.0 260.0,127.0","#5b5cf0",[43,887],{"x":888,"y":862,"width":863,"height":864,"rx":865,"fill":89,"stroke":90,"style":58},"276.0",[60,890,891],{"x":134,"y":417,"style":869},"transform('size')",[60,893,894],{"x":134,"y":873,"style":874},"count per group",[67,896],{"x1":897,"y1":879,"x2":898,"y2":879,"stroke":90,"style":58},"489.0","521.0",[882,900],{"points":901,"fill":885},"521.0,122.0 512.0,117.0 512.0,127.0",[43,903],{"x":904,"y":862,"width":863,"height":864,"rx":865,"fill":56,"stroke":57,"style":58},"528.0",[60,906,909],{"x":907,"y":417,"style":908},"632.0","font-size:14px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","aligned to rows",[60,911,912],{"x":907,"y":873,"style":874},"every row gets its group's",[60,914,916],{"x":907,"y":915,"style":874},"152","count",[60,918,920],{"x":134,"y":919,"style":136},"210","size counts rows; count counts non-null values in a column",[10,922,923,924,927,928,931,932,22],{},"A COUNTIF copied down a column — \"how many other rows share this row's region\" — is the same\n",[13,925,926],{},"transform"," pattern as the conditional sum, with ",[13,929,930],{},"size"," instead of ",[13,933,934],{},"sum",[144,936,938],{"className":172,"code":937,"language":174,"meta":149,"style":149},"sales[\"Orders in region\"] = sales.groupby(\"Region\")[\"Revenue\"].transform(\"size\")\n\ncounts = sales.groupby([\"Region\", \"Product\"]).size().reset_index(name=\"Orders\")\nprint(counts)\n",[13,939,940,970,974,1003],{"__ignoreMap":149},[153,941,942,945,948,950,952,955,957,960,962,965,968],{"class":67,"line":155},[153,943,944],{"class":185},"sales[",[153,946,947],{"class":162},"\"Orders in region\"",[153,949,544],{"class":185},[153,951,208],{"class":181},[153,953,954],{"class":185}," sales.groupby(",[153,956,541],{"class":162},[153,958,959],{"class":185},")[",[153,961,617],{"class":162},[153,963,964],{"class":185},"].transform(",[153,966,967],{"class":162},"\"size\"",[153,969,515],{"class":185},[153,971,972],{"class":67,"line":195},[153,973,199],{"emptyLinePlaceholder":198},[153,975,976,979,981,984,986,988,990,993,996,998,1001],{"class":67,"line":202},[153,977,978],{"class":185},"counts ",[153,980,208],{"class":181},[153,982,983],{"class":185}," sales.groupby([",[153,985,541],{"class":162},[153,987,226],{"class":185},[153,989,589],{"class":162},[153,991,992],{"class":185},"]).size().reset_index(",[153,994,995],{"class":496},"name",[153,997,208],{"class":181},[153,999,1000],{"class":162},"\"Orders\"",[153,1002,515],{"class":185},[153,1004,1005,1007],{"class":67,"line":214},[153,1006,535],{"class":343},[153,1008,1009],{"class":185},"(counts)\n",[10,1011,1012,1014,1015,1017],{},[13,1013,930],{}," counts every row in the group including nulls; ",[13,1016,916],{}," counts non-null values in the column\nyou apply it to. The difference is invisible on clean data and important on real exports, where a\ngroup of ten rows may have only seven populated revenue values.",[144,1019,1021],{"className":172,"code":1020,"language":174,"meta":149,"style":149},"per_region = sales.groupby(\"Region\").agg(\n    Rows=(\"Revenue\", \"size\"),\n    WithRevenue=(\"Revenue\", \"count\"),\n    Missing=(\"Revenue\", lambda s: s.isna().sum()),\n)\nprint(per_region)\n",[13,1022,1023,1037,1055,1073,1092,1096],{"__ignoreMap":149},[153,1024,1025,1028,1030,1032,1034],{"class":67,"line":155},[153,1026,1027],{"class":185},"per_region ",[153,1029,208],{"class":181},[153,1031,954],{"class":185},[153,1033,541],{"class":162},[153,1035,1036],{"class":185},").agg(\n",[153,1038,1039,1042,1044,1046,1048,1050,1052],{"class":67,"line":195},[153,1040,1041],{"class":496},"    Rows",[153,1043,208],{"class":181},[153,1045,674],{"class":185},[153,1047,617],{"class":162},[153,1049,226],{"class":185},[153,1051,967],{"class":162},[153,1053,1054],{"class":185},"),\n",[153,1056,1057,1060,1062,1064,1066,1068,1071],{"class":67,"line":202},[153,1058,1059],{"class":496},"    WithRevenue",[153,1061,208],{"class":181},[153,1063,674],{"class":185},[153,1065,617],{"class":162},[153,1067,226],{"class":185},[153,1069,1070],{"class":162},"\"count\"",[153,1072,1054],{"class":185},[153,1074,1075,1078,1080,1082,1084,1086,1089],{"class":67,"line":214},[153,1076,1077],{"class":496},"    Missing",[153,1079,208],{"class":181},[153,1081,674],{"class":185},[153,1083,617],{"class":162},[153,1085,226],{"class":185},[153,1087,1088],{"class":181},"lambda",[153,1090,1091],{"class":185}," s: s.isna().sum()),\n",[153,1093,1094],{"class":67,"line":256},[153,1095,515],{"class":185},[153,1097,1098,1100],{"class":67,"line":297},[153,1099,535],{"class":343},[153,1101,1102],{"class":185},"(per_region)\n",[10,1104,1105,1106,22],{},"That three-column view is the fastest way to find a group whose totals are quietly built on partial\ndata — the problem tackled directly in\n",[18,1107,1109],{"href":1108},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file\u002F","Find and Report Missing Values in an Excel File",[139,1111,1113],{"id":1112},"counting-distinct-values-within-groups","Counting distinct values within groups",[10,1115,1116],{},"This is the query that needs a genuinely unpleasant array formula in Excel and one call here.",[144,1118,1120],{"className":172,"code":1119,"language":174,"meta":149,"style":149},"print(sales.groupby(\"Region\")[\"Rep\"].nunique())\n\ndetail = sales.groupby(\"Region\").agg(\n    Reps=(\"Rep\", \"nunique\"),\n    Products=(\"Product\", \"nunique\"),\n    Orders=(\"Revenue\", \"size\"),\n)\nprint(detail)\n",[13,1121,1122,1138,1142,1155,1173,1190,1207,1211],{"__ignoreMap":149},[153,1123,1124,1126,1129,1131,1133,1135],{"class":67,"line":155},[153,1125,535],{"class":343},[153,1127,1128],{"class":185},"(sales.groupby(",[153,1130,541],{"class":162},[153,1132,959],{"class":185},[153,1134,746],{"class":162},[153,1136,1137],{"class":185},"].nunique())\n",[153,1139,1140],{"class":67,"line":195},[153,1141,199],{"emptyLinePlaceholder":198},[153,1143,1144,1147,1149,1151,1153],{"class":67,"line":202},[153,1145,1146],{"class":185},"detail ",[153,1148,208],{"class":181},[153,1150,954],{"class":185},[153,1152,541],{"class":162},[153,1154,1036],{"class":185},[153,1156,1157,1160,1162,1164,1166,1168,1171],{"class":67,"line":214},[153,1158,1159],{"class":496},"    Reps",[153,1161,208],{"class":181},[153,1163,674],{"class":185},[153,1165,746],{"class":162},[153,1167,226],{"class":185},[153,1169,1170],{"class":162},"\"nunique\"",[153,1172,1054],{"class":185},[153,1174,1175,1178,1180,1182,1184,1186,1188],{"class":67,"line":256},[153,1176,1177],{"class":496},"    Products",[153,1179,208],{"class":181},[153,1181,674],{"class":185},[153,1183,589],{"class":162},[153,1185,226],{"class":185},[153,1187,1170],{"class":162},[153,1189,1054],{"class":185},[153,1191,1192,1195,1197,1199,1201,1203,1205],{"class":67,"line":297},[153,1193,1194],{"class":496},"    Orders",[153,1196,208],{"class":181},[153,1198,674],{"class":185},[153,1200,617],{"class":162},[153,1202,226],{"class":185},[153,1204,967],{"class":162},[153,1206,1054],{"class":185},[153,1208,1209],{"class":67,"line":335},[153,1210,515],{"class":185},[153,1212,1213,1215],{"class":67,"line":379},[153,1214,535],{"class":343},[153,1216,1217],{"class":185},"(detail)\n",[10,1219,1220],{},"\"How many distinct reps sold in each region\" is a question every sales report eventually asks, and\nthe difficulty of expressing it in a spreadsheet is a good part of why so many reports do not.",[139,1222,1224],{"id":1223},"conditional-counts-with-wildcards-and-ranges","Conditional counts with wildcards and ranges",[144,1226,1228],{"className":172,"code":1227,"language":174,"meta":149,"style":149},"# =COUNTIF(C:C, \"Widg*\")\nprint(sales[\"Product\"].str.startswith(\"Widg\", na=False).sum())\n\n# =COUNTIFS(D:D, \">=5000\", D:D, \"\u003C12000\")\nband = sales[\"Revenue\"].between(5000, 12000, inclusive=\"left\")\nprint(band.sum())\n\n# Several acceptable values — Excel needs one COUNTIF per value\nprint(sales[\"Region\"].isin([\"North\", \"West\"]).sum())\n",[13,1229,1230,1235,1260,1264,1269,1303,1310,1314,1319],{"__ignoreMap":149},[153,1231,1232],{"class":67,"line":155},[153,1233,1234],{"class":529},"# =COUNTIF(C:C, \"Widg*\")\n",[153,1236,1237,1239,1241,1243,1246,1249,1251,1254,1256,1258],{"class":67,"line":195},[153,1238,535],{"class":343},[153,1240,729],{"class":185},[153,1242,589],{"class":162},[153,1244,1245],{"class":185},"].str.startswith(",[153,1247,1248],{"class":162},"\"Widg\"",[153,1250,226],{"class":185},[153,1252,1253],{"class":496},"na",[153,1255,208],{"class":181},[153,1257,512],{"class":343},[153,1259,553],{"class":185},[153,1261,1262],{"class":67,"line":202},[153,1263,199],{"emptyLinePlaceholder":198},[153,1265,1266],{"class":67,"line":214},[153,1267,1268],{"class":529},"# =COUNTIFS(D:D, \">=5000\", D:D, \"\u003C12000\")\n",[153,1270,1271,1274,1276,1278,1280,1283,1286,1288,1291,1293,1296,1298,1301],{"class":67,"line":256},[153,1272,1273],{"class":185},"band ",[153,1275,208],{"class":181},[153,1277,658],{"class":185},[153,1279,617],{"class":162},[153,1281,1282],{"class":185},"].between(",[153,1284,1285],{"class":343},"5000",[153,1287,226],{"class":185},[153,1289,1290],{"class":343},"12000",[153,1292,226],{"class":185},[153,1294,1295],{"class":496},"inclusive",[153,1297,208],{"class":181},[153,1299,1300],{"class":162},"\"left\"",[153,1302,515],{"class":185},[153,1304,1305,1307],{"class":67,"line":297},[153,1306,535],{"class":343},[153,1308,1309],{"class":185},"(band.sum())\n",[153,1311,1312],{"class":67,"line":335},[153,1313,199],{"emptyLinePlaceholder":198},[153,1315,1316],{"class":67,"line":379},[153,1317,1318],{"class":529},"# Several acceptable values — Excel needs one COUNTIF per value\n",[153,1320,1321,1323,1325,1327,1330,1332,1334,1336],{"class":67,"line":422},[153,1322,535],{"class":343},[153,1324,729],{"class":185},[153,1326,541],{"class":162},[153,1328,1329],{"class":185},"].isin([",[153,1331,223],{"class":162},[153,1333,226],{"class":185},[153,1335,238],{"class":162},[153,1337,1338],{"class":185},"]).sum())\n",[10,1340,1341,1344,1345,1347],{},[13,1342,1343],{},"between"," with an explicit ",[13,1346,1295],{}," argument is worth preferring over two comparisons: it states\nthe boundary behaviour in the code rather than leaving it to be inferred, which is the detail that\nmakes two people's counts disagree.",[139,1349,1351],{"id":1350},"building-a-data-quality-summary-from-counts","Building a data-quality summary from counts",[10,1353,1354],{},"Counting stops being a formula translation and starts being useful the moment several counts are put\nside by side. A short profile of every column — how many values, how many missing, how many distinct\n— is the first thing worth running against any workbook that arrives from somewhere else.",[144,1356,1358],{"className":172,"code":1357,"language":174,"meta":149,"style":149},"import pandas as pd\n\ndef profile(frame: pd.DataFrame) -> pd.DataFrame:\n    return pd.DataFrame({\n        \"dtype\": frame.dtypes.astype(str),\n        \"present\": frame.notna().sum(),\n        \"missing\": frame.isna().sum(),\n        \"distinct\": frame.nunique(dropna=True),\n        \"example\": [frame[c].dropna().iloc[0] if frame[c].notna().any() else None\n                    for c in frame.columns],\n    })\n\nprint(profile(sales))\n",[13,1359,1360,1370,1374,1386,1393,1406,1414,1422,1440,1464,1478,1483,1487],{"__ignoreMap":149},[153,1361,1362,1364,1366,1368],{"class":67,"line":155},[153,1363,182],{"class":181},[153,1365,186],{"class":185},[153,1367,189],{"class":181},[153,1369,192],{"class":185},[153,1371,1372],{"class":67,"line":195},[153,1373,199],{"emptyLinePlaceholder":198},[153,1375,1376,1379,1383],{"class":67,"line":202},[153,1377,1378],{"class":181},"def",[153,1380,1382],{"class":1381},"s_Opv"," profile",[153,1384,1385],{"class":185},"(frame: pd.DataFrame) -> pd.DataFrame:\n",[153,1387,1388,1391],{"class":67,"line":214},[153,1389,1390],{"class":181},"    return",[153,1392,211],{"class":185},[153,1394,1395,1398,1401,1404],{"class":67,"line":256},[153,1396,1397],{"class":162},"        \"dtype\"",[153,1399,1400],{"class":185},": frame.dtypes.astype(",[153,1402,1403],{"class":343},"str",[153,1405,1054],{"class":185},[153,1407,1408,1411],{"class":67,"line":297},[153,1409,1410],{"class":162},"        \"present\"",[153,1412,1413],{"class":185},": frame.notna().sum(),\n",[153,1415,1416,1419],{"class":67,"line":335},[153,1417,1418],{"class":162},"        \"missing\"",[153,1420,1421],{"class":185},": frame.isna().sum(),\n",[153,1423,1424,1427,1430,1433,1435,1438],{"class":67,"line":379},[153,1425,1426],{"class":162},"        \"distinct\"",[153,1428,1429],{"class":185},": frame.nunique(",[153,1431,1432],{"class":496},"dropna",[153,1434,208],{"class":181},[153,1436,1437],{"class":343},"True",[153,1439,1054],{"class":185},[153,1441,1442,1445,1448,1450,1452,1455,1458,1461],{"class":67,"line":422},[153,1443,1444],{"class":162},"        \"example\"",[153,1446,1447],{"class":185},": [frame[c].dropna().iloc[",[153,1449,45],{"class":343},[153,1451,544],{"class":185},[153,1453,1454],{"class":181},"if",[153,1456,1457],{"class":185}," frame[c].notna().any() ",[153,1459,1460],{"class":181},"else",[153,1462,1463],{"class":343}," None\n",[153,1465,1466,1469,1472,1475],{"class":67,"line":431},[153,1467,1468],{"class":181},"                    for",[153,1470,1471],{"class":185}," c ",[153,1473,1474],{"class":181},"in",[153,1476,1477],{"class":185}," frame.columns],\n",[153,1479,1480],{"class":67,"line":455},[153,1481,1482],{"class":185},"    })\n",[153,1484,1485],{"class":67,"line":473},[153,1486,199],{"emptyLinePlaceholder":198},[153,1488,1489,1491],{"class":67,"line":479},[153,1490,535],{"class":343},[153,1492,1493],{"class":185},"(profile(sales))\n",[10,1495,1496,1497,1500],{},"Two patterns in that table are worth reacting to immediately. A column whose distinct count equals\nits row count is an identifier, and should not be aggregated. A column whose distinct count is two or\nthree in a hundred thousand rows is a category, and converting it with ",[13,1498,1499],{},"astype(\"category\")"," will make\nevery later comparison faster and every group-by smaller.",[10,1502,1503,1504,1507,1508,1511,1512,22],{},"The profile is also the fastest way to spot the column that has quietly become text. A ",[13,1505,1506],{},"dtype"," of\n",[13,1509,1510],{},"object"," on something that should be numeric explains a total of zero long before anybody starts\ndoubting the logic, which is the diagnosis path set out in\n",[18,1513,1515],{"href":1514},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fcheck-excel-data-types-with-pandas\u002F","Check Excel Data Types with Pandas",[139,1517,1519],{"id":1518},"counting-into-a-report-someone-reads","Counting into a report someone reads",[10,1521,1522],{},"A count is rarely the deliverable on its own; it becomes one when it is framed against an expectation.\nWriting the count beside the number that was expected turns a figure into a check, and it costs one\nextra column.",[144,1524,1526],{"className":172,"code":1525,"language":174,"meta":149,"style":149},"expected = {\"North\": 3, \"South\": 2, \"West\": 2}\nactual = sales[\"Region\"].value_counts().to_dict()\n\ncheck = pd.DataFrame({\n    \"Region\": sorted(set(expected) | set(actual)),\n})\ncheck[\"Expected\"] = check[\"Region\"].map(expected).fillna(0).astype(int)\ncheck[\"Actual\"] = check[\"Region\"].map(actual).fillna(0).astype(int)\ncheck[\"Difference\"] = check[\"Actual\"] - check[\"Expected\"]\nprint(check)\n",[13,1527,1528,1566,1580,1584,1593,1619,1623,1653,1679,1706],{"__ignoreMap":149},[153,1529,1530,1533,1535,1538,1540,1543,1546,1548,1550,1552,1555,1557,1559,1561,1563],{"class":67,"line":155},[153,1531,1532],{"class":185},"expected ",[153,1534,208],{"class":181},[153,1536,1537],{"class":185}," {",[153,1539,223],{"class":162},[153,1541,1542],{"class":185},": ",[153,1544,1545],{"class":343},"3",[153,1547,226],{"class":185},[153,1549,229],{"class":162},[153,1551,1542],{"class":185},[153,1553,1554],{"class":343},"2",[153,1556,226],{"class":185},[153,1558,238],{"class":162},[153,1560,1542],{"class":185},[153,1562,1554],{"class":343},[153,1564,1565],{"class":185},"}\n",[153,1567,1568,1571,1573,1575,1577],{"class":67,"line":195},[153,1569,1570],{"class":185},"actual ",[153,1572,208],{"class":181},[153,1574,658],{"class":185},[153,1576,541],{"class":162},[153,1578,1579],{"class":185},"].value_counts().to_dict()\n",[153,1581,1582],{"class":67,"line":202},[153,1583,199],{"emptyLinePlaceholder":198},[153,1585,1586,1589,1591],{"class":67,"line":214},[153,1587,1588],{"class":185},"check ",[153,1590,208],{"class":181},[153,1592,211],{"class":185},[153,1594,1595,1597,1599,1602,1604,1607,1610,1613,1616],{"class":67,"line":256},[153,1596,217],{"class":162},[153,1598,1542],{"class":185},[153,1600,1601],{"class":343},"sorted",[153,1603,674],{"class":185},[153,1605,1606],{"class":343},"set",[153,1608,1609],{"class":185},"(expected) ",[153,1611,1612],{"class":181},"|",[153,1614,1615],{"class":343}," set",[153,1617,1618],{"class":185},"(actual)),\n",[153,1620,1621],{"class":67,"line":297},[153,1622,482],{"class":185},[153,1624,1625,1628,1631,1633,1635,1638,1640,1643,1645,1648,1651],{"class":67,"line":335},[153,1626,1627],{"class":185},"check[",[153,1629,1630],{"class":162},"\"Expected\"",[153,1632,544],{"class":185},[153,1634,208],{"class":181},[153,1636,1637],{"class":185}," check[",[153,1639,541],{"class":162},[153,1641,1642],{"class":185},"].map(expected).fillna(",[153,1644,45],{"class":343},[153,1646,1647],{"class":185},").astype(",[153,1649,1650],{"class":343},"int",[153,1652,515],{"class":185},[153,1654,1655,1657,1660,1662,1664,1666,1668,1671,1673,1675,1677],{"class":67,"line":379},[153,1656,1627],{"class":185},[153,1658,1659],{"class":162},"\"Actual\"",[153,1661,544],{"class":185},[153,1663,208],{"class":181},[153,1665,1637],{"class":185},[153,1667,541],{"class":162},[153,1669,1670],{"class":185},"].map(actual).fillna(",[153,1672,45],{"class":343},[153,1674,1647],{"class":185},[153,1676,1650],{"class":343},[153,1678,515],{"class":185},[153,1680,1681,1683,1686,1688,1690,1692,1694,1696,1699,1701,1703],{"class":67,"line":422},[153,1682,1627],{"class":185},[153,1684,1685],{"class":162},"\"Difference\"",[153,1687,544],{"class":185},[153,1689,208],{"class":181},[153,1691,1637],{"class":185},[153,1693,1659],{"class":162},[153,1695,544],{"class":185},[153,1697,1698],{"class":181},"-",[153,1700,1637],{"class":185},[153,1702,1630],{"class":162},[153,1704,1705],{"class":185},"]\n",[153,1707,1708,1710],{"class":67,"line":431},[153,1709,535],{"class":343},[153,1711,1712],{"class":185},"(check)\n",[10,1714,1715,1716,1719,1720,22],{},"Using ",[13,1717,1718],{},"set(expected) | set(actual)"," rather than either alone is the detail that makes this work: a\nregion that appears in the data but not the expectation, or vice versa, shows up as a row with a\ndifference rather than being silently dropped. That is the same reasoning behind the reconciliation\napproach in\n",[18,1721,1723],{"href":1722},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Ffind-rows-in-one-excel-file-missing-from-another\u002F","Find Rows in One Excel File Missing From Another",[139,1725,1727],{"id":1726},"common-pitfalls","Common pitfalls",[1729,1730,1731,1747],"table",{},[1732,1733,1734],"thead",{},[1735,1736,1737,1741,1744],"tr",{},[1738,1739,1740],"th",{},"Symptom",[1738,1742,1743],{},"Cause",[1738,1745,1746],{},"Fix",[1748,1749,1750,1766,1788,1803,1816,1832],"tbody",{},[1735,1751,1752,1756,1759],{},[1753,1754,1755],"td",{},"Count is higher than expected",[1753,1757,1758],{},"Empty strings counted as present",[1753,1760,1761,1762,1765],{},"Replace ",[13,1763,1764],{},"\"\""," with NaN before counting",[1735,1767,1768,1777,1785],{},[1753,1769,1770,1773,1774,1776],{},[13,1771,1772],{},"count()"," and ",[13,1775,930],{}," disagree",[1753,1778,1779,1781,1782,1784],{},[13,1780,916],{}," excludes nulls, ",[13,1783,930],{}," does not",[1753,1786,1787],{},"Choose deliberately; report both when investigating",[1735,1789,1790,1793,1796],{},[1753,1791,1792],{},"Distinct count differs from Excel",[1753,1794,1795],{},"Case or whitespace variants",[1753,1797,1798,1799,1802],{},"Normalise with ",[13,1800,1801],{},".str.strip().str.casefold()"," first",[1735,1804,1805,1808,1811],{},[1753,1806,1807],{},"Boundary values counted twice",[1753,1809,1810],{},"Two inclusive range conditions",[1753,1812,1813],{},[13,1814,1815],{},"between(..., inclusive=\"left\")",[1735,1817,1818,1823,1826],{},[1753,1819,1820,1822],{},[13,1821,799],{}," hides missing values",[1753,1824,1825],{},"It excludes NaN by default",[1753,1827,1828,1829],{},"Pass ",[13,1830,1831],{},"dropna=False",[1735,1833,1834,1837,1840],{},[1753,1835,1836],{},"A group is missing from the result",[1753,1838,1839],{},"Grouping key is NaN for those rows",[1753,1841,1842],{},[13,1843,1844],{},"groupby(..., dropna=False)",[139,1846,1848],{"id":1847},"counting-across-two-files","Counting across two files",[10,1850,1851],{},"Reconciliation questions are counting questions in disguise: how many invoices are in the ledger but\nnot the statement, how many appear twice, how many changed. All three come out of the same pair of\ncounts once both files are in frames.",[144,1853,1855],{"className":172,"code":1854,"language":174,"meta":149,"style":149},"ledger = pd.read_excel(\"ledger.xlsx\", dtype={\"Invoice\": \"string\"})\nstatement = pd.read_excel(\"statement.xlsx\", dtype={\"Invoice\": \"string\"})\n\nin_ledger = set(ledger[\"Invoice\"])\nin_statement = set(statement[\"Invoice\"])\n\nprint(f\"only in ledger    : {len(in_ledger - in_statement)}\")\nprint(f\"only in statement : {len(in_statement - in_ledger)}\")\nprint(f\"in both           : {len(in_ledger & in_statement)}\")\nprint(f\"duplicated in ledger: {ledger['Invoice'].duplicated().sum()}\")\n",[13,1856,1857,1888,1916,1920,1937,1953,1957,1987,2016,2043],{"__ignoreMap":149},[153,1858,1859,1862,1864,1867,1870,1872,1874,1876,1878,1881,1883,1886],{"class":67,"line":155},[153,1860,1861],{"class":185},"ledger ",[153,1863,208],{"class":181},[153,1865,1866],{"class":185}," pd.read_excel(",[153,1868,1869],{"class":162},"\"ledger.xlsx\"",[153,1871,226],{"class":185},[153,1873,1506],{"class":496},[153,1875,208],{"class":181},[153,1877,684],{"class":185},[153,1879,1880],{"class":162},"\"Invoice\"",[153,1882,1542],{"class":185},[153,1884,1885],{"class":162},"\"string\"",[153,1887,482],{"class":185},[153,1889,1890,1893,1895,1897,1900,1902,1904,1906,1908,1910,1912,1914],{"class":67,"line":195},[153,1891,1892],{"class":185},"statement ",[153,1894,208],{"class":181},[153,1896,1866],{"class":185},[153,1898,1899],{"class":162},"\"statement.xlsx\"",[153,1901,226],{"class":185},[153,1903,1506],{"class":496},[153,1905,208],{"class":181},[153,1907,684],{"class":185},[153,1909,1880],{"class":162},[153,1911,1542],{"class":185},[153,1913,1885],{"class":162},[153,1915,482],{"class":185},[153,1917,1918],{"class":67,"line":202},[153,1919,199],{"emptyLinePlaceholder":198},[153,1921,1922,1925,1927,1929,1932,1934],{"class":67,"line":214},[153,1923,1924],{"class":185},"in_ledger ",[153,1926,208],{"class":181},[153,1928,1615],{"class":343},[153,1930,1931],{"class":185},"(ledger[",[153,1933,1880],{"class":162},[153,1935,1936],{"class":185},"])\n",[153,1938,1939,1942,1944,1946,1949,1951],{"class":67,"line":256},[153,1940,1941],{"class":185},"in_statement ",[153,1943,208],{"class":181},[153,1945,1615],{"class":343},[153,1947,1948],{"class":185},"(statement[",[153,1950,1880],{"class":162},[153,1952,1936],{"class":185},[153,1954,1955],{"class":67,"line":297},[153,1956,199],{"emptyLinePlaceholder":198},[153,1958,1959,1961,1963,1965,1968,1970,1973,1976,1978,1981,1983,1985],{"class":67,"line":335},[153,1960,535],{"class":343},[153,1962,674],{"class":185},[153,1964,677],{"class":181},[153,1966,1967],{"class":162},"\"only in ledger    : ",[153,1969,684],{"class":683},[153,1971,1972],{"class":343},"len",[153,1974,1975],{"class":185},"(in_ledger ",[153,1977,1698],{"class":181},[153,1979,1980],{"class":185}," in_statement)",[153,1982,689],{"class":683},[153,1984,680],{"class":162},[153,1986,515],{"class":185},[153,1988,1989,1991,1993,1995,1998,2000,2002,2005,2007,2010,2012,2014],{"class":67,"line":379},[153,1990,535],{"class":343},[153,1992,674],{"class":185},[153,1994,677],{"class":181},[153,1996,1997],{"class":162},"\"only in statement : ",[153,1999,684],{"class":683},[153,2001,1972],{"class":343},[153,2003,2004],{"class":185},"(in_statement ",[153,2006,1698],{"class":181},[153,2008,2009],{"class":185}," in_ledger)",[153,2011,689],{"class":683},[153,2013,680],{"class":162},[153,2015,515],{"class":185},[153,2017,2018,2020,2022,2024,2027,2029,2031,2033,2035,2037,2039,2041],{"class":67,"line":422},[153,2019,535],{"class":343},[153,2021,674],{"class":185},[153,2023,677],{"class":181},[153,2025,2026],{"class":162},"\"in both           : ",[153,2028,684],{"class":683},[153,2030,1972],{"class":343},[153,2032,1975],{"class":185},[153,2034,583],{"class":181},[153,2036,1980],{"class":185},[153,2038,689],{"class":683},[153,2040,680],{"class":162},[153,2042,515],{"class":185},[153,2044,2045,2047,2049,2051,2054,2056,2059,2062,2065,2067,2069],{"class":67,"line":431},[153,2046,535],{"class":343},[153,2048,674],{"class":185},[153,2050,677],{"class":181},[153,2052,2053],{"class":162},"\"duplicated in ledger: ",[153,2055,684],{"class":683},[153,2057,2058],{"class":185},"ledger[",[153,2060,2061],{"class":162},"'Invoice'",[153,2063,2064],{"class":185},"].duplicated().sum()",[153,2066,689],{"class":683},[153,2068,680],{"class":162},[153,2070,515],{"class":185},[10,2072,2073,2074,2077],{},"Reading the key column as ",[13,2075,2076],{},"string"," rather than letting pandas infer it is what makes this reliable.\nAn invoice number that looks numeric becomes an integer in one file and a float in the other the\nmoment a single blank appears, and then the two sets share nothing at all — a mismatch that looks\ncatastrophic and is entirely an artefact of the read.",[10,2079,2080,2081,22],{},"The counts alone are usually enough to decide whether a difference is worth investigating. When it\nis, the row-level comparison is the natural next step, and it is covered in\n",[18,2082,2084],{"href":2083},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fcompare-two-excel-files-for-differences-with-python\u002F","Compare Two Excel Files for Differences with Python",[139,2086,2088],{"id":2087},"performance-and-scale","Performance and scale",[10,2090,2091],{},"Counting is the cheapest thing pandas does — a boolean mask is a compact array of bytes and summing\nit is a single pass. The cost, when there is one, comes from building the mask on a text column with\na regular expression, which is materially slower than an equality comparison.",[24,2093,33,2099,33,2102,33,2105,33,2109,33,2115,33,2122,33,2131,33,2137,33,2141,33,2143,33,2147,33,2152,33,2156,33,2159,33,2163,33,2168,33,2172,33,2175,33,2179,33,2183,33,2187],{"viewBox":2094,"role":27,"ariaLabelledBy":2095,"xmlns":31,"style":2098},"0 0 720 240",[2096,2097],"cif-cost-t","cif-cost-d","width:100%;max-width:720px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[35,2100,2101],{"id":2096},"Cost of building the mask, by comparison type",[39,2103,2104],{"id":2097},"An equality comparison on a category column is the cheapest, a literal substring search costs more, and a regular expression evaluated per row is the most expensive by a wide margin.",[43,2106],{"x":45,"y":45,"width":2107,"height":2108,"fill":48},"720","240",[60,2110,2114],{"x":2111,"y":2112,"style":2113},"20","56","font-size:12px;font-weight:600;fill:var(--text,#172033);text-anchor:start","regex contains",[43,2116],{"x":2117,"y":2118,"width":134,"height":52,"rx":2119,"fill":2120,"stroke":2121},"200","40","6","#e7ebef","var(--line,#cdd5e6)",[43,2123],{"x":2124,"y":2125,"width":2126,"height":2127,"rx":2128,"fill":2129,"stroke":2130},"201","41","378.0","24","5","#fee8f2","var(--accent,#d81b73)",[60,2132,2136],{"x":2133,"y":2134,"style":2135},"592.0","58","font-size:12px;font-weight:700;fill:var(--accent,#d81b73);text-anchor:start","per-row engine",[60,2138,2140],{"x":2111,"y":2139,"style":2113},"100","literal contains",[43,2142],{"x":2117,"y":75,"width":134,"height":52,"rx":2119,"fill":2120,"stroke":2121},[43,2144],{"x":2124,"y":2145,"width":2146,"height":2127,"rx":2128,"fill":112,"stroke":113},"85","126.0",[60,2148,2151],{"x":2133,"y":2149,"style":2150},"102","font-size:12px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:start","regex=False",[60,2153,2155],{"x":2111,"y":2154,"style":2113},"144","equality on strings",[43,2157],{"x":2117,"y":2158,"width":134,"height":52,"rx":2119,"fill":2120,"stroke":2121},"128",[43,2160],{"x":2124,"y":2161,"width":2162,"height":2127,"rx":2128,"fill":56,"stroke":57},"129","41.9",[60,2164,2167],{"x":2133,"y":2165,"style":2166},"146","font-size:12px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:start","direct compare",[60,2169,2171],{"x":2111,"y":2170,"style":2113},"188","equality on category",[43,2173],{"x":2117,"y":2174,"width":134,"height":52,"rx":2119,"fill":2120,"stroke":2121},"172",[43,2176],{"x":2124,"y":2177,"width":2178,"height":2127,"rx":2128,"fill":56,"stroke":57},"173","28",[60,2180,2182],{"x":2133,"y":2181,"style":2166},"190","integer codes",[60,2184,2186],{"x":2111,"y":2111,"style":2185},"font-size:11.5px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:start","relative cost",[60,2188,2191],{"x":2189,"y":2190,"style":136},"360.0","230","the count is free; building the mask is not",[144,2193,2195],{"className":172,"code":2194,"language":174,"meta":149,"style":149},"# Fast: exact comparison on a categorical or string column\nexact = (sales[\"Product\"] == \"Widget\").sum()\n\n# Slower: a regex evaluated per row\npattern = sales[\"Product\"].str.contains(r\"^Widg\\w+$\", na=False, regex=True).sum()\n\n# Fast again: fixed substring, no regex engine\nliteral = sales[\"Product\"].str.contains(\"Widg\", na=False, regex=False).sum()\n",[13,2196,2197,2202,2222,2226,2231,2286,2290,2295],{"__ignoreMap":149},[153,2198,2199],{"class":67,"line":155},[153,2200,2201],{"class":529},"# Fast: exact comparison on a categorical or string column\n",[153,2203,2204,2207,2209,2211,2213,2215,2217,2219],{"class":67,"line":195},[153,2205,2206],{"class":185},"exact ",[153,2208,208],{"class":181},[153,2210,586],{"class":185},[153,2212,589],{"class":162},[153,2214,544],{"class":185},[153,2216,547],{"class":181},[153,2218,596],{"class":162},[153,2220,2221],{"class":185},").sum()\n",[153,2223,2224],{"class":67,"line":202},[153,2225,199],{"emptyLinePlaceholder":198},[153,2227,2228],{"class":67,"line":214},[153,2229,2230],{"class":529},"# Slower: a regex evaluated per row\n",[153,2232,2233,2236,2238,2240,2242,2245,2248,2250,2253,2256,2259,2262,2265,2267,2269,2271,2273,2275,2277,2280,2282,2284],{"class":67,"line":256},[153,2234,2235],{"class":185},"pattern ",[153,2237,208],{"class":181},[153,2239,658],{"class":185},[153,2241,589],{"class":162},[153,2243,2244],{"class":185},"].str.contains(",[153,2246,2247],{"class":181},"r",[153,2249,680],{"class":162},[153,2251,2252],{"class":343},"^",[153,2254,2255],{"class":162},"Widg",[153,2257,2258],{"class":343},"\\w",[153,2260,2261],{"class":181},"+",[153,2263,2264],{"class":343},"$",[153,2266,680],{"class":162},[153,2268,226],{"class":185},[153,2270,1253],{"class":496},[153,2272,208],{"class":181},[153,2274,512],{"class":343},[153,2276,226],{"class":185},[153,2278,2279],{"class":496},"regex",[153,2281,208],{"class":181},[153,2283,1437],{"class":343},[153,2285,2221],{"class":185},[153,2287,2288],{"class":67,"line":297},[153,2289,199],{"emptyLinePlaceholder":198},[153,2291,2292],{"class":67,"line":335},[153,2293,2294],{"class":529},"# Fast again: fixed substring, no regex engine\n",[153,2296,2297,2300,2302,2304,2306,2308,2310,2312,2314,2316,2318,2320,2322,2324,2326],{"class":67,"line":379},[153,2298,2299],{"class":185},"literal ",[153,2301,208],{"class":181},[153,2303,658],{"class":185},[153,2305,589],{"class":162},[153,2307,2244],{"class":185},[153,2309,1248],{"class":162},[153,2311,226],{"class":185},[153,2313,1253],{"class":496},[153,2315,208],{"class":181},[153,2317,512],{"class":343},[153,2319,226],{"class":185},[153,2321,2279],{"class":496},[153,2323,208],{"class":181},[153,2325,512],{"class":343},[153,2327,2221],{"class":185},[10,2329,2330,2331,2333,2334,2337],{},"Passing ",[13,2332,2151],{}," when the pattern is a literal substring is a free improvement that people\nrarely make. On a column with hundreds of thousands of rows, converting a repeated-value text column\nto ",[13,2335,2336],{},"category"," dtype first is another — comparisons then operate on integer codes rather than strings.",[139,2339,2341],{"id":2340},"conclusion","Conclusion",[10,2343,2344,2346,2347,2350,2351,2353,2354,2357,2358,2360],{},[13,2345,15],{}," is a boolean mask summed, ",[13,2348,2349],{},"COUNTIFS"," is the same with conditions joined by ",[13,2352,583],{},", and a\ncopied-down COUNTIF is ",[13,2355,2356],{},"groupby().transform(\"size\")",". Beyond the direct translations, pandas answers\nquestions Excel makes hard: distinct counts per group in one call, a full value breakdown with\n",[13,2359,799],{},", and the proportion matching from the same mask that produced the count.",[139,2362,2364],{"id":2363},"frequently-asked-questions","Frequently asked questions",[10,2366,2367,2371],{},[2368,2369,2370],"strong",{},"Why does mask.sum() count rows?","\nBecause a boolean Series sums as ones and zeros: True is 1 and False is 0, so the sum is the number of True values. mask.mean() gives the proportion instead, which is the share of rows matching.",[10,2373,2374,2377,2378,2381,2382,2384],{},[2368,2375,2376],{},"What is the equivalent of COUNTA and COUNTBLANK?","\nCOUNTA is df",[153,2379,2380],{},"col",".notna().sum() (or .count(), which excludes NaN by default), and COUNTBLANK is df",[153,2383,2380],{},".isna().sum(). Watch for empty strings read from a sheet, which are not NaN and are counted as present.",[10,2386,2387,2390,2391,2393,2394,2396],{},[2368,2388,2389],{},"How do I count distinct values?","\ndf",[153,2392,2380],{},".nunique() for the count, and df",[153,2395,2380],{},".value_counts() for the breakdown. Excel needs an array formula with SUMPRODUCT and COUNTIF to do the same thing, which is one of the clearest wins in this translation.",[10,2398,2399,2402,2403,2406],{},[2368,2400,2401],{},"Does COUNTIFS with an OR condition translate?","\nExcel needs several COUNTIFS added together for an OR. pandas uses the | operator, or .isin(",[153,2404,2405],{},"...",") when the condition is membership in a list — one expression either way.",[139,2408,2410],{"id":2409},"related","Related",[2412,2413,2414,2421,2428,2435,2440],"ul",{},[2415,2416,2417,2418,2420],"li",{},"Up one level: ",[18,2419,21],{"href":20}," — the wider function map.",[2415,2422,2423,2427],{},[18,2424,2426],{"href":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fsumif-and-sumifs-equivalent-in-pandas\u002F","SUMIF and SUMIFS Equivalent in pandas"," — the same masks, summed instead of counted.",[2415,2429,2430,2434],{},[18,2431,2433],{"href":2432},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Ffind-duplicate-rows-in-excel-with-python\u002F","Find Duplicate Rows in Excel with Python"," — counting repeats as a data-quality check.",[2415,2436,2437,2439],{},[18,2438,1109],{"href":1108}," — the blank-counting problem in full.",[2415,2441,2442,2444],{},[18,2443,833],{"href":832}," — why two counts of the same column disagree.",[2446,2447,2448],"style",{},"html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-wDw, html code.shiki .s-wDw{--shiki-default:#6A737D;--shiki-dark:#BDC4CC}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}",{"title":149,"searchDepth":195,"depth":195,"links":2450},[2451,2452,2453,2454,2455,2456,2457,2458,2459,2460,2461,2462,2463,2464],{"id":141,"depth":195,"text":142},{"id":518,"depth":195,"text":519},{"id":713,"depth":195,"text":714},{"id":837,"depth":195,"text":838},{"id":1112,"depth":195,"text":1113},{"id":1223,"depth":195,"text":1224},{"id":1350,"depth":195,"text":1351},{"id":1518,"depth":195,"text":1519},{"id":1726,"depth":195,"text":1727},{"id":1847,"depth":195,"text":1848},{"id":2087,"depth":195,"text":2088},{"id":2340,"depth":195,"text":2341},{"id":2363,"depth":195,"text":2364},{"id":2409,"depth":195,"text":2410},"2026-09-04","Count matching rows by summing a boolean mask, count blanks and distinct values in one call, and replace a copied-down COUNTIF with groupby().transform('size').","md",[2469,2471,2473,2475],{"q":2370,"a":2470},"Because a boolean Series sums as ones and zeros: True is 1 and False is 0, so the sum is the number of True values. mask.mean() gives the proportion instead, which is the share of rows matching.",{"q":2376,"a":2472},"COUNTA is df[col].notna().sum() (or .count(), which excludes NaN by default), and COUNTBLANK is df[col].isna().sum(). Watch for empty strings read from a sheet, which are not NaN and are counted as present.",{"q":2389,"a":2474},"df[col].nunique() for the count, and df[col].value_counts() for the breakdown. Excel needs an array formula with SUMPRODUCT and COUNTIF to do the same thing, which is one of the clearest wins in this translation.",{"q":2401,"a":2476},"Excel needs several COUNTIFS added together for an OR. pandas uses the | operator, or .isin([...]) when the condition is membership in a list — one expression either way.",{"breadcrumb":2478},[2479,2482,2485],{"name":2480,"item":2481},"Home","\u002F",{"name":2483,"item":2484},"Advanced Data Transformation and Cleaning","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":21,"item":20},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fcountif-and-countifs-equivalent-in-pandas",{"title":5,"description":2488},"Translate COUNTIF, COUNTIFS, COUNTA, COUNTBLANK and distinct counts into pandas — boolean masks, value_counts, nunique per group, and the wildcard equivalents.","countif-and-countifs-equivalent-in-pandas","advanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fcountif-and-countifs-equivalent-in-pandas\u002Findex","how-to","2j7Ma4_fR_v0oUZblX3y3zV0YUAz1uuWBsxkveaA2J0",[2494,2497],{"title":21,"path":2495,"stem":2496,"children":-1},"\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas","advanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Findex",{"title":2498,"path":2499,"stem":2500,"children":-1},"Excel IF Formulas as pandas Conditional Columns","\u002Fadvanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fexcel-if-formulas-as-pandas-conditional-columns","advanced-data-transformation-and-cleaning\u002Fexcel-formula-equivalents-in-pandas\u002Fexcel-if-formulas-as-pandas-conditional-columns\u002Findex",1788710154434]