[{"data":1,"prerenderedAt":2460},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas":2427,"ancestors:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas":2436,"children:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas":2441,"related:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas":2442},{"id":4,"title":5,"body":6,"dateModified":2393,"datePublished":2393,"description":2394,"extension":2395,"faq":2396,"keyword":2405,"kind":2406,"meta":2407,"navigation":107,"path":2416,"seo":2417,"slug":2420,"stem":2421,"topics":2422,"type":2425,"__hash__":2426},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas\u002Findex.md","Profile an Excel Dataset with pandas",{"type":7,"value":8,"toc":2380},"minimark",[9,37,42,73,79,159,177,333,337,340,595,617,621,628,826,863,867,878,1188,1217,1302,1306,1309,1577,1583,1587,1590,1691,1712,1716,1719,1996,1999,2260,2264,2308,2312,2315,2319,2325,2331,2337,2343,2347,2376],[10,11,12,13,17,18,21,22,25,26,30,31,36],"p",{},"The first hour with a new spreadsheet export decides how the rest of the work goes. Which columns are really numbers? Which are mostly empty? Is ",[14,15,16],"code",{},"CustomerID"," unique, or does the same customer appear twice? Does the ",[14,19,20],{},"Postcode"," column hold postcodes, or also phone numbers typed in the wrong cell? ",[14,23,24],{},"df.describe()"," answers a fraction of that for numeric columns only. A ",[27,28,29],"em",{},"profile"," answers it for every column in one table, and writing it to Excel turns it into something to discuss with whoever owns the data. This page, part of ",[32,33,35],"a",{"href":34},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002F","Validating Excel Data with Python",", builds a profile function step by step and runs it on a realistic export.",[38,39,41],"h2",{"id":40},"prerequisites","Prerequisites",[43,44,49],"pre",{"className":45,"code":46,"language":47,"meta":48,"style":48},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl xlsxwriter\n","bash","",[14,50,51],{"__ignoreMap":48},[52,53,56,60,64,67,70],"span",{"class":54,"line":55},"line",1,[52,57,59],{"class":58},"sMTad","pip",[52,61,63],{"class":62},"srMev"," install",[52,65,66],{"class":62}," pandas",[52,68,69],{"class":62}," openpyxl",[52,71,72],{"class":62}," xlsxwriter\n",[10,74,75,78],{},[14,76,77],{},"customers_export.xlsx"," has 500 customers with an ID, name, segment, postcode, join date, annual spend, employee count and a legacy code. The export has problems that are typical and not obvious by scrolling: a duplicated ID, a segment spelled two ways, postcodes mixed with a phone number and a placeholder, two spend values stored as text, a mostly empty employee column and a legacy column that is almost entirely blank.",[43,80,84],{"className":81,"code":82,"language":83,"meta":48,"style":48},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\ndf = pd.read_excel(\"customers_export.xlsx\")\nprint(df.shape)\nprint(df.dtypes.to_dict())\nprint(df.describe().round(1))\n","python",[14,85,86,102,109,127,137,145],{"__ignoreMap":48},[52,87,88,92,96,99],{"class":54,"line":55},[52,89,91],{"class":90},"s-kum","import",[52,93,95],{"class":94},"skGVy"," pandas ",[52,97,98],{"class":90},"as",[52,100,101],{"class":94}," pd\n",[52,103,105],{"class":54,"line":104},2,[52,106,108],{"emptyLinePlaceholder":107},true,"\n",[52,110,112,115,118,121,124],{"class":54,"line":111},3,[52,113,114],{"class":94},"df ",[52,116,117],{"class":90},"=",[52,119,120],{"class":94}," pd.read_excel(",[52,122,123],{"class":62},"\"customers_export.xlsx\"",[52,125,126],{"class":94},")\n",[52,128,130,134],{"class":54,"line":129},4,[52,131,133],{"class":132},"sP0c6","print",[52,135,136],{"class":94},"(df.shape)\n",[52,138,140,142],{"class":54,"line":139},5,[52,141,133],{"class":132},[52,143,144],{"class":94},"(df.dtypes.to_dict())\n",[52,146,148,150,153,156],{"class":54,"line":147},6,[52,149,133],{"class":132},[52,151,152],{"class":94},"(df.describe().round(",[52,154,155],{"class":132},"1",[52,157,158],{"class":94},"))\n",[10,160,161,164,165,168,169,172,173,176],{},[14,162,163],{},"describe()"," covers the numeric and date columns — here ",[14,166,167],{},"Employees"," and ",[14,170,171],{},"Joined"," — and says nothing about the text columns, which is where most of the problems are. ",[14,174,175],{},"AnnualSpend"," is missing from it altogether: a text value in the column made pandas read the whole column as generic objects instead of numbers.",[178,179,187,188,187,192,187,196,187,204,187,214,187,225,187,231,187,237,187,243,187,251,187,255,187,262,187,266,187,271,187,275,187,280,187,284,187,291,187,296,187,300,187,304,187,309,187,312,187,317,187,320,187,325,187,328],"svg",{"viewBox":180,"role":181,"ariaLabelledBy":182,"xmlns":185,"style":186},"4 0 752 233","img",[183,184],"pr-describe-t","pr-describe-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[189,190,191],"title",{"id":183},"What describe() shows and what a profile adds",[193,194,195],"desc",{"id":184},"Two panels. describe() gives count, mean, spread and quartiles for numeric columns only. A profile gives, for every column, the type, blanks, distinct values, top values, mixed types, patterns and whether it could be a key.",[197,198],"rect",{"x":199,"y":200,"width":201,"height":202,"fill":203},"4","0","752","233","#ffffff",[197,205],{"x":206,"y":207,"width":208,"height":209,"rx":210,"fill":211,"stroke":212,"style":213},"20.0","16.0","351.0","201.0","14","#e7ebef","var(--line,#cdd5e6)","stroke-width:2px",[215,216,218],"g",{"transform":217},"translate(182.5,29.0) scale(1.083)",[219,220],"path",{"d":221,"fill":222,"stroke":223,"style":224},"M4 20h16 M7 17v-5 M12 17V7 M17 17v-8","none","var(--text,#172033)","stroke-width:1.66px;stroke-linecap:round;stroke-linejoin:round",[226,227,24],"text",{"x":228,"y":229,"style":230},"195.5","78.0","font-size:14px;font-weight:700;fill:var(--text,#172033);text-anchor:middle",[54,232],{"x1":233,"y1":234,"x2":235,"y2":234,"stroke":212,"style":236},"36.0","91.0","355.0","stroke-width:1px",[226,238,242],{"x":239,"y":240,"style":241},"60.0","111.0","font-size:12px;font-weight:400;fill:var(--text,#172033);text-anchor:start","count, mean, std, quartiles",[215,244,246],{"transform":245},"translate(38.5,99.5) scale(0.625)",[219,247],{"d":248,"fill":222,"stroke":249,"style":250},"M5 12.5l4.5 4.5L19 7","var(--teal,#0f9488)","stroke-width:2.88px;stroke-linecap:round;stroke-linejoin:round",[226,252,254],{"x":239,"y":253,"style":241},"137.0","numeric columns only",[215,256,258],{"transform":257},"translate(38.5,125.5) scale(0.625)",[219,259],{"d":260,"fill":222,"stroke":261,"style":250},"M6 6l12 12 M18 6L6 18","var(--accent,#f43f8f)",[226,263,265],{"x":239,"y":264,"style":241},"163.0","no blanks share, no top values",[215,267,269],{"transform":268},"translate(38.5,151.5) scale(0.625)",[219,270],{"d":260,"fill":222,"stroke":261,"style":250},[226,272,274],{"x":239,"y":273,"style":241},"189.0","no mixed-type warning",[215,276,278],{"transform":277},"translate(38.5,177.5) scale(0.625)",[219,279],{"d":260,"fill":222,"stroke":261,"style":250},[197,281],{"x":282,"y":207,"width":208,"height":209,"rx":210,"fill":283,"stroke":249,"style":213},"389.0","#d9f4f1",[215,285,287],{"transform":286},"translate(551.5,29.0) scale(1.083)",[219,288],{"d":289,"fill":222,"stroke":290,"style":224},"M11 4a7 7 0 1 0 .01 0z M16 16l5 5","var(--teal-ink,#0b6157)",[226,292,295],{"x":293,"y":229,"style":294},"564.5","font-size:14px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","A column profile",[54,297],{"x1":298,"y1":234,"x2":299,"y2":234,"stroke":249,"style":236},"405.0","724.0",[226,301,303],{"x":302,"y":240,"style":241},"429.0","every column, one row each",[215,305,307],{"transform":306},"translate(407.5,99.5) scale(0.625)",[219,308],{"d":248,"fill":222,"stroke":249,"style":250},[226,310,311],{"x":302,"y":253,"style":241},"blanks, distinct, top values",[215,313,315],{"transform":314},"translate(407.5,125.5) scale(0.625)",[219,316],{"d":248,"fill":222,"stroke":249,"style":250},[226,318,319],{"x":302,"y":264,"style":241},"mixed types and patterns",[215,321,323],{"transform":322},"translate(407.5,151.5) scale(0.625)",[219,324],{"d":248,"fill":222,"stroke":249,"style":250},[226,326,327],{"x":302,"y":273,"style":241},"candidate keys and duplicates",[215,329,331],{"transform":330},"translate(407.5,177.5) scale(0.625)",[219,332],{"d":248,"fill":222,"stroke":249,"style":250},[38,334,336],{"id":335},"step-1-the-basic-per-column-facts","Step 1 — The basic per-column facts",[10,338,339],{},"Start with what is true of any column: its type, how many values are missing, how many are distinct, and which values are most common:",[43,341,343],{"className":81,"code":342,"language":83,"meta":48,"style":48},"def basic_profile(df):\n    rows = []\n    for col in df.columns:\n        s = df[col]\n        top = s.value_counts(dropna=True).head(3)\n        rows.append({\n            \"column\": col,\n            \"dtype\": str(s.dtype),\n            \"missing %\": round(s.isna().mean() * 100, 1),\n            \"distinct\": s.nunique(dropna=True),\n            \"top values\": \", \".join(f\"{v} ({c})\" for v, c in top.items()),\n        })\n    return pd.DataFrame(rows)\n\nprofile = basic_profile(df)\nprint(profile.to_string(index=False))\n",[14,344,345,357,367,381,391,418,423,432,447,475,492,546,552,561,566,577],{"__ignoreMap":48},[52,346,347,350,354],{"class":54,"line":55},[52,348,349],{"class":90},"def",[52,351,353],{"class":352},"s_Opv"," basic_profile",[52,355,356],{"class":94},"(df):\n",[52,358,359,362,364],{"class":54,"line":104},[52,360,361],{"class":94},"    rows ",[52,363,117],{"class":90},[52,365,366],{"class":94}," []\n",[52,368,369,372,375,378],{"class":54,"line":111},[52,370,371],{"class":90},"    for",[52,373,374],{"class":94}," col ",[52,376,377],{"class":90},"in",[52,379,380],{"class":94}," df.columns:\n",[52,382,383,386,388],{"class":54,"line":129},[52,384,385],{"class":94},"        s ",[52,387,117],{"class":90},[52,389,390],{"class":94}," df[col]\n",[52,392,393,396,398,401,405,407,410,413,416],{"class":54,"line":139},[52,394,395],{"class":94},"        top ",[52,397,117],{"class":90},[52,399,400],{"class":94}," s.value_counts(",[52,402,404],{"class":403},"sa561","dropna",[52,406,117],{"class":90},[52,408,409],{"class":132},"True",[52,411,412],{"class":94},").head(",[52,414,415],{"class":132},"3",[52,417,126],{"class":94},[52,419,420],{"class":54,"line":147},[52,421,422],{"class":94},"        rows.append({\n",[52,424,426,429],{"class":54,"line":425},7,[52,427,428],{"class":62},"            \"column\"",[52,430,431],{"class":94},": col,\n",[52,433,435,438,441,444],{"class":54,"line":434},8,[52,436,437],{"class":62},"            \"dtype\"",[52,439,440],{"class":94},": ",[52,442,443],{"class":132},"str",[52,445,446],{"class":94},"(s.dtype),\n",[52,448,450,453,455,458,461,464,467,470,472],{"class":54,"line":449},9,[52,451,452],{"class":62},"            \"missing %\"",[52,454,440],{"class":94},[52,456,457],{"class":132},"round",[52,459,460],{"class":94},"(s.isna().mean() ",[52,462,463],{"class":90},"*",[52,465,466],{"class":132}," 100",[52,468,469],{"class":94},", ",[52,471,155],{"class":132},[52,473,474],{"class":94},"),\n",[52,476,478,481,484,486,488,490],{"class":54,"line":477},10,[52,479,480],{"class":62},"            \"distinct\"",[52,482,483],{"class":94},": s.nunique(",[52,485,404],{"class":403},[52,487,117],{"class":90},[52,489,409],{"class":132},[52,491,474],{"class":94},[52,493,495,498,500,503,506,509,512,516,519,522,525,527,530,532,535,538,541,543],{"class":54,"line":494},11,[52,496,497],{"class":62},"            \"top values\"",[52,499,440],{"class":94},[52,501,502],{"class":62},"\", \"",[52,504,505],{"class":94},".join(",[52,507,508],{"class":90},"f",[52,510,511],{"class":62},"\"",[52,513,515],{"class":514},"sSjpA","{",[52,517,518],{"class":94},"v",[52,520,521],{"class":514},"}",[52,523,524],{"class":62}," (",[52,526,515],{"class":514},[52,528,529],{"class":94},"c",[52,531,521],{"class":514},[52,533,534],{"class":62},")\"",[52,536,537],{"class":90}," for",[52,539,540],{"class":94}," v, c ",[52,542,377],{"class":90},[52,544,545],{"class":94}," top.items()),\n",[52,547,549],{"class":54,"line":548},12,[52,550,551],{"class":94},"        })\n",[52,553,555,558],{"class":54,"line":554},13,[52,556,557],{"class":90},"    return",[52,559,560],{"class":94}," pd.DataFrame(rows)\n",[52,562,564],{"class":54,"line":563},14,[52,565,108],{"emptyLinePlaceholder":107},[52,567,569,572,574],{"class":54,"line":568},15,[52,570,571],{"class":94},"profile ",[52,573,117],{"class":90},[52,575,576],{"class":94}," basic_profile(df)\n",[52,578,580,582,585,588,590,593],{"class":54,"line":579},16,[52,581,133],{"class":132},[52,583,584],{"class":94},"(profile.to_string(",[52,586,587],{"class":403},"index",[52,589,117],{"class":90},[52,591,592],{"class":132},"False",[52,594,158],{"class":94},[10,596,597,598,601,602,604,605,608,609,612,613,616],{},"The table already tells a story. ",[14,599,600],{},"Legacy code"," is over 99% empty and has two distinct values — a candidate for deletion. ",[14,603,167],{}," is 58% empty, which matters if anything downstream divides by it. ",[14,606,607],{},"Segment"," has four distinct values where three were expected: ",[14,610,611],{},"business"," in lower case next to ",[14,614,615],{},"Business",". Postcodes are 8% missing.",[38,618,620],{"id":619},"step-2-detect-columns-that-mix-types","Step 2 — Detect columns that mix types",[10,622,623,624,627],{},"A column with numbers and a few stray strings is read as text, and every numeric operation on it fails or — worse — sorts ",[14,625,626],{},"\"1,250.00\""," among the text. Count the Python types present in each column:",[43,629,631],{"className":81,"code":630,"language":83,"meta":48,"style":48},"def type_mix(s):\n    kinds = s.dropna().map(lambda v: type(v).__name__).value_counts()\n    return \", \".join(f\"{k}: {c}\" for k, c in kinds.items())\n\nprofile[\"types\"] = [type_mix(df[c]) for c in df.columns]\nmixed = profile[profile[\"types\"].str.contains(\",\")]\nprint(mixed[[\"column\", \"types\"]].to_string(index=False))\nprint(df.loc[df[\"AnnualSpend\"].map(lambda v: isinstance(v, str)), [\"CustomerID\", \"AnnualSpend\"]])\n",[14,632,633,643,671,711,715,742,763,788],{"__ignoreMap":48},[52,634,635,637,640],{"class":54,"line":55},[52,636,349],{"class":90},[52,638,639],{"class":352}," type_mix",[52,641,642],{"class":94},"(s):\n",[52,644,645,648,650,653,656,659,662,665,668],{"class":54,"line":104},[52,646,647],{"class":94},"    kinds ",[52,649,117],{"class":90},[52,651,652],{"class":94}," s.dropna().map(",[52,654,655],{"class":90},"lambda",[52,657,658],{"class":94}," v: ",[52,660,661],{"class":132},"type",[52,663,664],{"class":94},"(v).",[52,666,667],{"class":132},"__name__",[52,669,670],{"class":94},").value_counts()\n",[52,672,673,675,678,680,682,684,686,689,691,693,695,697,699,701,703,706,708],{"class":54,"line":111},[52,674,557],{"class":90},[52,676,677],{"class":62}," \", \"",[52,679,505],{"class":94},[52,681,508],{"class":90},[52,683,511],{"class":62},[52,685,515],{"class":514},[52,687,688],{"class":94},"k",[52,690,521],{"class":514},[52,692,440],{"class":62},[52,694,515],{"class":514},[52,696,529],{"class":94},[52,698,521],{"class":514},[52,700,511],{"class":62},[52,702,537],{"class":90},[52,704,705],{"class":94}," k, c ",[52,707,377],{"class":90},[52,709,710],{"class":94}," kinds.items())\n",[52,712,713],{"class":54,"line":129},[52,714,108],{"emptyLinePlaceholder":107},[52,716,717,720,723,726,728,731,734,737,739],{"class":54,"line":139},[52,718,719],{"class":94},"profile[",[52,721,722],{"class":62},"\"types\"",[52,724,725],{"class":94},"] ",[52,727,117],{"class":90},[52,729,730],{"class":94}," [type_mix(df[c]) ",[52,732,733],{"class":90},"for",[52,735,736],{"class":94}," c ",[52,738,377],{"class":90},[52,740,741],{"class":94}," df.columns]\n",[52,743,744,747,749,752,754,757,760],{"class":54,"line":147},[52,745,746],{"class":94},"mixed ",[52,748,117],{"class":90},[52,750,751],{"class":94}," profile[profile[",[52,753,722],{"class":62},[52,755,756],{"class":94},"].str.contains(",[52,758,759],{"class":62},"\",\"",[52,761,762],{"class":94},")]\n",[52,764,765,767,770,773,775,777,780,782,784,786],{"class":54,"line":425},[52,766,133],{"class":132},[52,768,769],{"class":94},"(mixed[[",[52,771,772],{"class":62},"\"column\"",[52,774,469],{"class":94},[52,776,722],{"class":62},[52,778,779],{"class":94},"]].to_string(",[52,781,587],{"class":403},[52,783,117],{"class":90},[52,785,592],{"class":132},[52,787,158],{"class":94},[52,789,790,792,795,798,801,803,805,808,811,813,816,819,821,823],{"class":54,"line":434},[52,791,133],{"class":132},[52,793,794],{"class":94},"(df.loc[df[",[52,796,797],{"class":62},"\"AnnualSpend\"",[52,799,800],{"class":94},"].map(",[52,802,655],{"class":90},[52,804,658],{"class":94},[52,806,807],{"class":132},"isinstance",[52,809,810],{"class":94},"(v, ",[52,812,443],{"class":132},[52,814,815],{"class":94},")), [",[52,817,818],{"class":62},"\"CustomerID\"",[52,820,469],{"class":94},[52,822,797],{"class":62},[52,824,825],{"class":94},"]])\n",[10,827,828,830,831,834,835,838,839,842,843,469,845,469,848,851,852,855,856,858,859,862],{},[14,829,175],{}," holds mostly floats, three whole numbers and one string, ",[14,832,833],{},"1,250.00"," — a value formatted as text by whoever typed it. The profile names the cell, which turns \"why won't this column sum?\" into a specific fix. The export also had an ",[14,836,837],{},"n\u002Fa"," in that column, and it does not show up: ",[14,840,841],{},"read_excel"," treats ",[14,844,837],{},[14,846,847],{},"NA",[14,849,850],{},"null"," and a few similar strings as missing by default, so it arrived as a blank. That is convenient for loading but bad for profiling, because the placeholder is itself a finding. To see such values, read with ",[14,853,854],{},"keep_default_na=False"," for the profile run. ",[14,857,20],{}," holds only strings, so it does not show up here — its problem is the ",[27,860,861],{},"shape"," of the strings, which the next step looks at.",[38,864,866],{"id":865},"step-3-summarise-the-shape-of-the-values","Step 3 — Summarise the shape of the values",[10,868,869,870,873,874,877],{},"Replace every letter with ",[14,871,872],{},"A"," and every digit with ",[14,875,876],{},"9"," and count the resulting patterns. Well-behaved columns have one or two patterns; anything else stands out:",[43,879,881],{"className":81,"code":880,"language":83,"meta":48,"style":48},"import re\n\ndef shape(v):\n    return re.sub(r\"[0-9]\", \"9\", re.sub(r\"[A-Za-z]\", \"A\", str(v)))\n\ndef patterns(s, n=4):\n    counts = s.dropna().map(shape).value_counts()\n    return \", \".join(f\"{p} ({c})\" for p, c in counts.head(n).items()) + (f\" … +{len(counts) - n}\" if len(counts) > n else \"\")\n\nprofile[\"patterns\"] = [patterns(df[c]) if df[c].dtype == object or str(df[c].dtype).startswith(\"str\") else \"\" for c in df.columns]\nprint(profile.loc[profile[\"patterns\"] != \"\", [\"column\", \"patterns\"]].to_string(index=False))\nprint(df[\"Postcode\"].dropna().map(shape).value_counts().tail(3))\n",[14,882,883,890,894,904,960,964,981,991,1079,1083,1136,1171],{"__ignoreMap":48},[52,884,885,887],{"class":54,"line":55},[52,886,91],{"class":90},[52,888,889],{"class":94}," re\n",[52,891,892],{"class":54,"line":104},[52,893,108],{"emptyLinePlaceholder":107},[52,895,896,898,901],{"class":54,"line":111},[52,897,349],{"class":90},[52,899,900],{"class":352}," shape",[52,902,903],{"class":94},"(v):\n",[52,905,906,908,911,914,916,919,922,925,927,929,932,935,937,939,941,944,946,948,950,953,955,957],{"class":54,"line":129},[52,907,557],{"class":90},[52,909,910],{"class":94}," re.sub(",[52,912,913],{"class":90},"r",[52,915,511],{"class":62},[52,917,918],{"class":132},"[",[52,920,921],{"class":514},"0-9",[52,923,924],{"class":132},"]",[52,926,511],{"class":62},[52,928,469],{"class":94},[52,930,931],{"class":62},"\"9\"",[52,933,934],{"class":94},", re.sub(",[52,936,913],{"class":90},[52,938,511],{"class":62},[52,940,918],{"class":132},[52,942,943],{"class":514},"A-Za-z",[52,945,924],{"class":132},[52,947,511],{"class":62},[52,949,469],{"class":94},[52,951,952],{"class":62},"\"A\"",[52,954,469],{"class":94},[52,956,443],{"class":132},[52,958,959],{"class":94},"(v)))\n",[52,961,962],{"class":54,"line":139},[52,963,108],{"emptyLinePlaceholder":107},[52,965,966,968,971,974,976,978],{"class":54,"line":147},[52,967,349],{"class":90},[52,969,970],{"class":352}," patterns",[52,972,973],{"class":94},"(s, n",[52,975,117],{"class":90},[52,977,199],{"class":132},[52,979,980],{"class":94},"):\n",[52,982,983,986,988],{"class":54,"line":425},[52,984,985],{"class":94},"    counts ",[52,987,117],{"class":90},[52,989,990],{"class":94}," s.dropna().map(shape).value_counts()\n",[52,992,993,995,997,999,1001,1003,1005,1007,1009,1011,1013,1015,1017,1019,1021,1024,1026,1029,1032,1034,1036,1039,1041,1044,1047,1050,1053,1055,1057,1060,1063,1065,1068,1071,1074,1077],{"class":54,"line":434},[52,994,557],{"class":90},[52,996,677],{"class":62},[52,998,505],{"class":94},[52,1000,508],{"class":90},[52,1002,511],{"class":62},[52,1004,515],{"class":514},[52,1006,10],{"class":94},[52,1008,521],{"class":514},[52,1010,524],{"class":62},[52,1012,515],{"class":514},[52,1014,529],{"class":94},[52,1016,521],{"class":514},[52,1018,534],{"class":62},[52,1020,537],{"class":90},[52,1022,1023],{"class":94}," p, c ",[52,1025,377],{"class":90},[52,1027,1028],{"class":94}," counts.head(n).items()) ",[52,1030,1031],{"class":90},"+",[52,1033,524],{"class":94},[52,1035,508],{"class":90},[52,1037,1038],{"class":62},"\" … +",[52,1040,515],{"class":514},[52,1042,1043],{"class":132},"len",[52,1045,1046],{"class":94},"(counts) ",[52,1048,1049],{"class":90},"-",[52,1051,1052],{"class":94}," n",[52,1054,521],{"class":514},[52,1056,511],{"class":62},[52,1058,1059],{"class":90}," if",[52,1061,1062],{"class":132}," len",[52,1064,1046],{"class":94},[52,1066,1067],{"class":90},">",[52,1069,1070],{"class":94}," n ",[52,1072,1073],{"class":90},"else",[52,1075,1076],{"class":62}," \"\"",[52,1078,126],{"class":94},[52,1080,1081],{"class":54,"line":449},[52,1082,108],{"emptyLinePlaceholder":107},[52,1084,1085,1087,1090,1092,1094,1097,1100,1103,1106,1109,1112,1115,1118,1121,1124,1126,1128,1130,1132,1134],{"class":54,"line":477},[52,1086,719],{"class":94},[52,1088,1089],{"class":62},"\"patterns\"",[52,1091,725],{"class":94},[52,1093,117],{"class":90},[52,1095,1096],{"class":94}," [patterns(df[c]) ",[52,1098,1099],{"class":90},"if",[52,1101,1102],{"class":94}," df[c].dtype ",[52,1104,1105],{"class":90},"==",[52,1107,1108],{"class":132}," object",[52,1110,1111],{"class":90}," or",[52,1113,1114],{"class":132}," str",[52,1116,1117],{"class":94},"(df[c].dtype).startswith(",[52,1119,1120],{"class":62},"\"str\"",[52,1122,1123],{"class":94},") ",[52,1125,1073],{"class":90},[52,1127,1076],{"class":62},[52,1129,537],{"class":90},[52,1131,736],{"class":94},[52,1133,377],{"class":90},[52,1135,741],{"class":94},[52,1137,1138,1140,1143,1145,1147,1150,1152,1155,1157,1159,1161,1163,1165,1167,1169],{"class":54,"line":494},[52,1139,133],{"class":132},[52,1141,1142],{"class":94},"(profile.loc[profile[",[52,1144,1089],{"class":62},[52,1146,725],{"class":94},[52,1148,1149],{"class":90},"!=",[52,1151,1076],{"class":62},[52,1153,1154],{"class":94},", [",[52,1156,772],{"class":62},[52,1158,469],{"class":94},[52,1160,1089],{"class":62},[52,1162,779],{"class":94},[52,1164,587],{"class":403},[52,1166,117],{"class":90},[52,1168,592],{"class":132},[52,1170,158],{"class":94},[52,1172,1173,1175,1178,1181,1184,1186],{"class":54,"line":548},[52,1174,133],{"class":132},[52,1176,1177],{"class":94},"(df[",[52,1179,1180],{"class":62},"\"Postcode\"",[52,1182,1183],{"class":94},"].dropna().map(shape).value_counts().tail(",[52,1185,415],{"class":132},[52,1187,158],{"class":94},[10,1189,1190,1192,1193,1196,1197,1199,1200,1203,1204,1207,1208,1211,1212,1216],{},[14,1191,16],{}," has a single pattern, ",[14,1194,1195],{},"AA-99999",", as an ID should. ",[14,1198,20],{}," has the expected UK shapes plus a few that do not belong: ",[14,1201,1202],{},"99999 999999"," (a phone number), ",[14,1205,1206],{},"AAAAAAA"," (the word \"unknown\") and ",[14,1209,1210],{},"A99AA"," (a postcode typed without its space). Patterns find this kind of problem without anyone having to write a validation rule first; once found, the rules can be written — see ",[32,1213,1215],{"href":1214},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fvalidate-email-and-phone-formats-in-excel-data\u002F","Validate Email and Phone Formats in Excel Data",".",[178,1218,187,1224,187,1227,187,1230,187,1234,187,1239,187,1245,187,1252,187,1256,187,1259,187,1263,187,1265,187,1267,187,1270,187,1274,187,1278,187,1281,187,1284,187,1288,187,1291,187,1296],{"viewBox":1219,"role":181,"ariaLabelledBy":1220,"xmlns":185,"style":1223},"4 0 744 206",[1221,1222],"pr-patterns-t","pr-patterns-d","width:100%;max-width:744px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[189,1225,1226],{"id":1221},"Postcode value shapes, least common first",[193,1228,1229],{"id":1222},"Horizontal bars counting the rarest value shapes in the postcode column. A phone number shape, a word and a postcode without its space each appear once, far below the normal postcode shapes.",[197,1231],{"x":199,"y":200,"width":1232,"height":1233,"fill":203},"744","206",[197,1235],{"x":206,"y":207,"width":1236,"height":1237,"rx":1238,"fill":211,"stroke":212,"style":236},"178.3","28.0","7",[226,1240,1244],{"x":1241,"y":1242,"style":1243},"109.2","34.5","font-size:12.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","99999 999999 (phone)",[197,1246],{"x":1247,"y":207,"width":1248,"height":1237,"rx":1238,"fill":1249,"stroke":1250,"style":1251},"208.3","6.0","#fce9e9","var(--danger,#dc2626)","stroke-width:1.5px",[226,1253,155],{"x":1254,"y":1242,"style":1255},"222.3","font-size:12px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:start",[197,1257],{"x":206,"y":1258,"width":1236,"height":1237,"rx":1238,"fill":211,"stroke":212,"style":236},"56.0",[226,1260,1262],{"x":1241,"y":1261,"style":1243},"74.5","AAAAAAA (word)",[197,1264],{"x":1247,"y":1258,"width":1248,"height":1237,"rx":1238,"fill":1249,"stroke":1250,"style":1251},[226,1266,155],{"x":1254,"y":1261,"style":1255},[197,1268],{"x":206,"y":1269,"width":1236,"height":1237,"rx":1238,"fill":211,"stroke":212,"style":236},"96.0",[226,1271,1273],{"x":1241,"y":1272,"style":1243},"114.5","A99AA (no space)",[197,1275],{"x":1247,"y":1269,"width":1248,"height":1237,"rx":1238,"fill":1276,"stroke":1277,"style":1251},"#fdefd8","var(--gold,#b4740a)",[226,1279,155],{"x":1254,"y":1272,"style":1280},"font-size:12px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:start",[197,1282],{"x":206,"y":1283,"width":1236,"height":1237,"rx":1238,"fill":211,"stroke":212,"style":236},"136.0",[226,1285,1287],{"x":1241,"y":1286,"style":1243},"154.5","A9 9AA (valid)",[197,1289],{"x":1247,"y":1283,"width":1290,"height":1237,"rx":1238,"fill":283,"stroke":249,"style":1251},"493.9",[226,1292,1295],{"x":1293,"y":1286,"style":1294},"710.3","font-size:12px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:start","193",[226,1297,1301],{"x":1298,"y":1299,"style":1300},"380.0","186.0","font-size:12.5px;font-weight:400;fill:var(--muted,#5b6780);text-anchor:middle","Rare shapes are almost always errors; the full list is in the profile.",[38,1303,1305],{"id":1304},"step-4-numbers-dates-and-text-lengths","Step 4 — Numbers, dates and text lengths",[10,1307,1308],{},"For numeric and date columns add the range; for text columns the length range, which catches truncated values and stray long entries:",[43,1310,1312],{"className":81,"code":1311,"language":83,"meta":48,"style":48},"def ranges(s):\n    num = pd.to_numeric(s, errors=\"coerce\") if s.dtype == object else s\n    if pd.api.types.is_numeric_dtype(num) and num.notna().any():\n        return f\"{num.min():,.2f} to {num.max():,.2f}\"\n    if pd.api.types.is_datetime64_any_dtype(s):\n        return f\"{s.min():%Y-%m-%d} to {s.max():%Y-%m-%d}\"\n    lengths = s.dropna().astype(str).str.len()\n    return f\"length {lengths.min()} to {lengths.max()}\" if len(lengths) else \"\"\n\nprofile[\"range\"] = [ranges(df[c]) for c in df.columns]\nprint(profile[[\"column\", \"range\"]].to_string(index=False))\n",[14,1313,1314,1323,1358,1372,1407,1414,1474,1489,1528,1532,1554],{"__ignoreMap":48},[52,1315,1316,1318,1321],{"class":54,"line":55},[52,1317,349],{"class":90},[52,1319,1320],{"class":352}," ranges",[52,1322,642],{"class":94},[52,1324,1325,1328,1330,1333,1336,1338,1341,1343,1345,1348,1350,1352,1355],{"class":54,"line":104},[52,1326,1327],{"class":94},"    num ",[52,1329,117],{"class":90},[52,1331,1332],{"class":94}," pd.to_numeric(s, ",[52,1334,1335],{"class":403},"errors",[52,1337,117],{"class":90},[52,1339,1340],{"class":62},"\"coerce\"",[52,1342,1123],{"class":94},[52,1344,1099],{"class":90},[52,1346,1347],{"class":94}," s.dtype ",[52,1349,1105],{"class":90},[52,1351,1108],{"class":132},[52,1353,1354],{"class":90}," else",[52,1356,1357],{"class":94}," s\n",[52,1359,1360,1363,1366,1369],{"class":54,"line":111},[52,1361,1362],{"class":90},"    if",[52,1364,1365],{"class":94}," pd.api.types.is_numeric_dtype(num) ",[52,1367,1368],{"class":90},"and",[52,1370,1371],{"class":94}," num.notna().any():\n",[52,1373,1374,1377,1380,1382,1384,1387,1390,1392,1395,1397,1400,1402,1404],{"class":54,"line":129},[52,1375,1376],{"class":90},"        return",[52,1378,1379],{"class":90}," f",[52,1381,511],{"class":62},[52,1383,515],{"class":514},[52,1385,1386],{"class":94},"num.min()",[52,1388,1389],{"class":90},":,.2f",[52,1391,521],{"class":514},[52,1393,1394],{"class":62}," to ",[52,1396,515],{"class":514},[52,1398,1399],{"class":94},"num.max()",[52,1401,1389],{"class":90},[52,1403,521],{"class":514},[52,1405,1406],{"class":62},"\"\n",[52,1408,1409,1411],{"class":54,"line":139},[52,1410,1362],{"class":90},[52,1412,1413],{"class":94}," pd.api.types.is_datetime64_any_dtype(s):\n",[52,1415,1416,1418,1420,1422,1424,1427,1430,1433,1436,1438,1441,1443,1446,1448,1450,1452,1455,1457,1460,1463,1466,1468,1470,1472],{"class":54,"line":147},[52,1417,1376],{"class":90},[52,1419,1379],{"class":90},[52,1421,511],{"class":62},[52,1423,515],{"class":514},[52,1425,1426],{"class":94},"s.min()",[52,1428,1429],{"class":90},":",[52,1431,1432],{"class":94},"%",[52,1434,1435],{"class":90},"Y-",[52,1437,1432],{"class":94},[52,1439,1440],{"class":90},"m-",[52,1442,1432],{"class":94},[52,1444,1445],{"class":90},"d",[52,1447,521],{"class":514},[52,1449,1394],{"class":62},[52,1451,515],{"class":514},[52,1453,1454],{"class":94},"s.max():",[52,1456,1432],{"class":90},[52,1458,1459],{"class":94},"Y",[52,1461,1462],{"class":90},"-%",[52,1464,1465],{"class":94},"m",[52,1467,1462],{"class":90},[52,1469,1445],{"class":94},[52,1471,521],{"class":514},[52,1473,1406],{"class":62},[52,1475,1476,1479,1481,1484,1486],{"class":54,"line":425},[52,1477,1478],{"class":94},"    lengths ",[52,1480,117],{"class":90},[52,1482,1483],{"class":94}," s.dropna().astype(",[52,1485,443],{"class":132},[52,1487,1488],{"class":94},").str.len()\n",[52,1490,1491,1493,1495,1498,1500,1503,1505,1507,1509,1512,1514,1516,1518,1520,1523,1525],{"class":54,"line":434},[52,1492,557],{"class":90},[52,1494,1379],{"class":90},[52,1496,1497],{"class":62},"\"length ",[52,1499,515],{"class":514},[52,1501,1502],{"class":94},"lengths.min()",[52,1504,521],{"class":514},[52,1506,1394],{"class":62},[52,1508,515],{"class":514},[52,1510,1511],{"class":94},"lengths.max()",[52,1513,521],{"class":514},[52,1515,511],{"class":62},[52,1517,1059],{"class":90},[52,1519,1062],{"class":132},[52,1521,1522],{"class":94},"(lengths) ",[52,1524,1073],{"class":90},[52,1526,1527],{"class":62}," \"\"\n",[52,1529,1530],{"class":54,"line":449},[52,1531,108],{"emptyLinePlaceholder":107},[52,1533,1534,1536,1539,1541,1543,1546,1548,1550,1552],{"class":54,"line":477},[52,1535,719],{"class":94},[52,1537,1538],{"class":62},"\"range\"",[52,1540,725],{"class":94},[52,1542,117],{"class":90},[52,1544,1545],{"class":94}," [ranges(df[c]) ",[52,1547,733],{"class":90},[52,1549,736],{"class":94},[52,1551,377],{"class":90},[52,1553,741],{"class":94},[52,1555,1556,1558,1561,1563,1565,1567,1569,1571,1573,1575],{"class":54,"line":494},[52,1557,133],{"class":132},[52,1559,1560],{"class":94},"(profile[[",[52,1562,772],{"class":62},[52,1564,469],{"class":94},[52,1566,1538],{"class":62},[52,1568,779],{"class":94},[52,1570,587],{"class":403},[52,1572,117],{"class":90},[52,1574,592],{"class":132},[52,1576,158],{"class":94},[10,1578,1579,1580,1582],{},"For a mixed column like ",[14,1581,175],{},", the range is computed on the values that do convert, so it still gives a sense of the scale. The join dates span January 2019 to February 2026, which is plausible; a date in 1900 or in the future would be worth a question.",[38,1584,1586],{"id":1585},"step-5-candidate-keys-and-duplicates","Step 5 — Candidate keys and duplicates",[10,1588,1589],{},"A column that should identify each row must be complete and unique. Test every column, and list the duplicates of the one that should be the key:",[43,1591,1593],{"className":81,"code":1592,"language":83,"meta":48,"style":48},"profile[\"unique\"] = [df[c].notna().all() and df[c].is_unique for c in df.columns]\nprint(profile.loc[profile[\"unique\"], \"column\"].tolist())\ndupes = df[df[\"CustomerID\"].duplicated(keep=False)]\nprint(dupes[[\"CustomerID\", \"Name\", \"Segment\"]].to_string(index=False))\n",[14,1594,1595,1622,1638,1662],{"__ignoreMap":48},[52,1596,1597,1599,1602,1604,1606,1609,1611,1614,1616,1618,1620],{"class":54,"line":55},[52,1598,719],{"class":94},[52,1600,1601],{"class":62},"\"unique\"",[52,1603,725],{"class":94},[52,1605,117],{"class":90},[52,1607,1608],{"class":94}," [df[c].notna().all() ",[52,1610,1368],{"class":90},[52,1612,1613],{"class":94}," df[c].is_unique ",[52,1615,733],{"class":90},[52,1617,736],{"class":94},[52,1619,377],{"class":90},[52,1621,741],{"class":94},[52,1623,1624,1626,1628,1630,1633,1635],{"class":54,"line":104},[52,1625,133],{"class":132},[52,1627,1142],{"class":94},[52,1629,1601],{"class":62},[52,1631,1632],{"class":94},"], ",[52,1634,772],{"class":62},[52,1636,1637],{"class":94},"].tolist())\n",[52,1639,1640,1643,1645,1648,1650,1653,1656,1658,1660],{"class":54,"line":111},[52,1641,1642],{"class":94},"dupes ",[52,1644,117],{"class":90},[52,1646,1647],{"class":94}," df[df[",[52,1649,818],{"class":62},[52,1651,1652],{"class":94},"].duplicated(",[52,1654,1655],{"class":403},"keep",[52,1657,117],{"class":90},[52,1659,592],{"class":132},[52,1661,762],{"class":94},[52,1663,1664,1666,1669,1671,1673,1676,1678,1681,1683,1685,1687,1689],{"class":54,"line":129},[52,1665,133],{"class":132},[52,1667,1668],{"class":94},"(dupes[[",[52,1670,818],{"class":62},[52,1672,469],{"class":94},[52,1674,1675],{"class":62},"\"Name\"",[52,1677,469],{"class":94},[52,1679,1680],{"class":62},"\"Segment\"",[52,1682,779],{"class":94},[52,1684,587],{"class":403},[52,1686,117],{"class":90},[52,1688,592],{"class":132},[52,1690,158],{"class":94},[10,1692,1693,1695,1696,1699,1700,1703,1704,1706,1707,1711],{},[14,1694,16],{}," is not unique: ",[14,1697,1698],{},"CU-10037"," appears twice, attached to two different names. ",[14,1701,1702],{},"Name"," is unique here, but a name is never a good key. Before this export is loaded anywhere with ",[14,1705,16],{}," as the primary key, someone has to decide which of the two records is right — ",[32,1708,1710],{"href":1709},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Ffind-duplicate-rows-in-excel-with-python\u002F","Find Duplicate Rows in Excel with Python"," covers the options.",[38,1713,1715],{"id":1714},"step-6-write-the-profile-to-excel","Step 6 — Write the profile to Excel",[10,1717,1718],{},"Put the profile on the first sheet, with the worst columns easy to spot, and each column's full value counts on a second sheet:",[43,1720,1722],{"className":81,"code":1721,"language":83,"meta":48,"style":48},"with pd.ExcelWriter(\"customers_profile.xlsx\", engine=\"xlsxwriter\") as xw:\n    profile.to_excel(xw, sheet_name=\"Profile\", index=False)\n    wb, ws = xw.book, xw.sheets[\"Profile\"]\n    ws.set_column(\"A:B\", 14)\n    ws.set_column(\"C:D\", 10)\n    ws.set_column(\"E:H\", 42)\n    ws.conditional_format(1, 2, len(profile), 2, {\"type\": \"data_bar\", \"bar_color\": \"#F59E0B\", \"bar_solid\": True})\n    ws.freeze_panes(1, 1)\n    counts = pd.concat({c: df[c].astype(str).value_counts().head(20) for c in df.columns}, names=[\"column\", \"value\"])\n    counts.rename(\"count\").to_frame().to_excel(xw, sheet_name=\"Top values\")\nprint(profile[[\"column\", \"missing %\", \"distinct\", \"unique\"]].to_string(index=False))\n",[14,1723,1724,1752,1775,1790,1804,1818,1832,1886,1899,1944,1964],{"__ignoreMap":48},[52,1725,1726,1729,1732,1735,1737,1740,1742,1745,1747,1749],{"class":54,"line":55},[52,1727,1728],{"class":90},"with",[52,1730,1731],{"class":94}," pd.ExcelWriter(",[52,1733,1734],{"class":62},"\"customers_profile.xlsx\"",[52,1736,469],{"class":94},[52,1738,1739],{"class":403},"engine",[52,1741,117],{"class":90},[52,1743,1744],{"class":62},"\"xlsxwriter\"",[52,1746,1123],{"class":94},[52,1748,98],{"class":90},[52,1750,1751],{"class":94}," xw:\n",[52,1753,1754,1757,1760,1762,1765,1767,1769,1771,1773],{"class":54,"line":104},[52,1755,1756],{"class":94},"    profile.to_excel(xw, ",[52,1758,1759],{"class":403},"sheet_name",[52,1761,117],{"class":90},[52,1763,1764],{"class":62},"\"Profile\"",[52,1766,469],{"class":94},[52,1768,587],{"class":403},[52,1770,117],{"class":90},[52,1772,592],{"class":132},[52,1774,126],{"class":94},[52,1776,1777,1780,1782,1785,1787],{"class":54,"line":111},[52,1778,1779],{"class":94},"    wb, ws ",[52,1781,117],{"class":90},[52,1783,1784],{"class":94}," xw.book, xw.sheets[",[52,1786,1764],{"class":62},[52,1788,1789],{"class":94},"]\n",[52,1791,1792,1795,1798,1800,1802],{"class":54,"line":129},[52,1793,1794],{"class":94},"    ws.set_column(",[52,1796,1797],{"class":62},"\"A:B\"",[52,1799,469],{"class":94},[52,1801,210],{"class":132},[52,1803,126],{"class":94},[52,1805,1806,1808,1811,1813,1816],{"class":54,"line":139},[52,1807,1794],{"class":94},[52,1809,1810],{"class":62},"\"C:D\"",[52,1812,469],{"class":94},[52,1814,1815],{"class":132},"10",[52,1817,126],{"class":94},[52,1819,1820,1822,1825,1827,1830],{"class":54,"line":147},[52,1821,1794],{"class":94},[52,1823,1824],{"class":62},"\"E:H\"",[52,1826,469],{"class":94},[52,1828,1829],{"class":132},"42",[52,1831,126],{"class":94},[52,1833,1834,1837,1839,1841,1844,1846,1848,1851,1853,1856,1859,1861,1864,1866,1869,1871,1874,1876,1879,1881,1883],{"class":54,"line":425},[52,1835,1836],{"class":94},"    ws.conditional_format(",[52,1838,155],{"class":132},[52,1840,469],{"class":94},[52,1842,1843],{"class":132},"2",[52,1845,469],{"class":94},[52,1847,1043],{"class":132},[52,1849,1850],{"class":94},"(profile), ",[52,1852,1843],{"class":132},[52,1854,1855],{"class":94},", {",[52,1857,1858],{"class":62},"\"type\"",[52,1860,440],{"class":94},[52,1862,1863],{"class":62},"\"data_bar\"",[52,1865,469],{"class":94},[52,1867,1868],{"class":62},"\"bar_color\"",[52,1870,440],{"class":94},[52,1872,1873],{"class":62},"\"#F59E0B\"",[52,1875,469],{"class":94},[52,1877,1878],{"class":62},"\"bar_solid\"",[52,1880,440],{"class":94},[52,1882,409],{"class":132},[52,1884,1885],{"class":94},"})\n",[52,1887,1888,1891,1893,1895,1897],{"class":54,"line":434},[52,1889,1890],{"class":94},"    ws.freeze_panes(",[52,1892,155],{"class":132},[52,1894,469],{"class":94},[52,1896,155],{"class":132},[52,1898,126],{"class":94},[52,1900,1901,1903,1905,1908,1910,1913,1916,1918,1920,1922,1924,1927,1930,1932,1934,1936,1938,1941],{"class":54,"line":449},[52,1902,985],{"class":94},[52,1904,117],{"class":90},[52,1906,1907],{"class":94}," pd.concat({c: df[c].astype(",[52,1909,443],{"class":132},[52,1911,1912],{"class":94},").value_counts().head(",[52,1914,1915],{"class":132},"20",[52,1917,1123],{"class":94},[52,1919,733],{"class":90},[52,1921,736],{"class":94},[52,1923,377],{"class":90},[52,1925,1926],{"class":94}," df.columns}, ",[52,1928,1929],{"class":403},"names",[52,1931,117],{"class":90},[52,1933,918],{"class":94},[52,1935,772],{"class":62},[52,1937,469],{"class":94},[52,1939,1940],{"class":62},"\"value\"",[52,1942,1943],{"class":94},"])\n",[52,1945,1946,1949,1952,1955,1957,1959,1962],{"class":54,"line":477},[52,1947,1948],{"class":94},"    counts.rename(",[52,1950,1951],{"class":62},"\"count\"",[52,1953,1954],{"class":94},").to_frame().to_excel(xw, ",[52,1956,1759],{"class":403},[52,1958,117],{"class":90},[52,1960,1961],{"class":62},"\"Top values\"",[52,1963,126],{"class":94},[52,1965,1966,1968,1970,1972,1974,1977,1979,1982,1984,1986,1988,1990,1992,1994],{"class":54,"line":494},[52,1967,133],{"class":132},[52,1969,1560],{"class":94},[52,1971,772],{"class":62},[52,1973,469],{"class":94},[52,1975,1976],{"class":62},"\"missing %\"",[52,1978,469],{"class":94},[52,1980,1981],{"class":62},"\"distinct\"",[52,1983,469],{"class":94},[52,1985,1601],{"class":62},[52,1987,779],{"class":94},[52,1989,587],{"class":403},[52,1991,117],{"class":90},[52,1993,592],{"class":132},[52,1995,158],{"class":94},[10,1997,1998],{},"A data bar on the missing-percentage column makes the emptiest columns jump out. The profile is small, readable and specific — the right thing to send to the owner of the export with the question \"which of these are expected?\"",[178,2000,187,2006,187,2009,187,2012,187,2017,187,2022,187,2027,187,2034,187,2039,187,2044,187,2047,187,2052,187,2056,187,2060,187,2064,187,2068,187,2072,187,2076,187,2080,187,2084,187,2087,187,2091,187,2095,187,2099,187,2101,187,2104,187,2106,187,2109,187,2111,187,2114,187,2116,187,2119,187,2122,187,2125,187,2127,187,2130,187,2132,187,2134,187,2136,187,2139,187,2141,187,2144,187,2146,187,2150,187,2153,187,2156,187,2158,187,2160,187,2162,187,2164,187,2166,187,2168,187,2170,187,2172,187,2174,187,2176,187,2179,187,2182,187,2184,187,2186,187,2188,187,2190,187,2192,187,2194,187,2196,187,2198,187,2200,187,2202,187,2205,187,2208,187,2210,187,2212,187,2214,187,2217,187,2219,187,2223,187,2225,187,2228,187,2230,187,2232,187,2235,187,2239,187,2241,187,2243,187,2245,187,2247,187,2249,187,2252,187,2254,187,2256,187,2258],{"viewBox":2001,"role":181,"ariaLabelledBy":2002,"xmlns":185,"style":2005},"161 0 438 254",[2003,2004],"pr-report-t","pr-report-d","width:100%;max-width:438px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[189,2007,2008],{"id":2003},"The Profile sheet",[193,2010,2011],{"id":2004},"A worksheet with one row per column of the export, showing its type, missing percentage, number of distinct values and whether it is unique. The legacy code column is over 99 percent empty and CustomerID is not unique.",[197,2013],{"x":2014,"y":200,"width":2015,"height":2016,"fill":203},"161","438","254",[197,2018],{"x":2019,"y":207,"width":298,"height":2020,"rx":2021,"fill":211,"stroke":212,"style":1251},"177.5","30.0","8",[197,2023],{"x":2019,"y":2024,"width":298,"height":2025,"rx":200,"fill":211,"stroke":212,"style":2026},"38.0","8.0","stroke-width:0px",[215,2028,2030],{"transform":2029},"translate(187.5,23.0) scale(0.667)",[219,2031],{"d":2032,"fill":222,"stroke":290,"style":2033},"M6 3h9l4 4v14H6z M15 3v4h4 M9.5 11.5l4.5 6 M14 11.5l-4.5 6","stroke-width:2.70px;stroke-linecap:round;stroke-linejoin:round",[226,2035,2038],{"x":2036,"y":233,"style":2037},"209.5","font-size:12px;font-weight:700;fill:var(--text,#172033);text-anchor:start","customers_profile.xlsx",[197,2040],{"x":2019,"y":2041,"width":2042,"height":2043,"rx":200,"fill":211,"stroke":212,"style":236},"46.0","32.0","24.0",[197,2045],{"x":2036,"y":2041,"width":2046,"height":2043,"rx":200,"fill":211,"stroke":212,"style":236},"95.9",[226,2048,872],{"x":2049,"y":2050,"style":2051},"257.5","63.0","font-size:11px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle",[197,2053],{"x":2054,"y":2041,"width":2055,"height":2043,"rx":200,"fill":211,"stroke":212,"style":236},"305.4","65.0",[226,2057,2059],{"x":2058,"y":2050,"style":2051},"338.0","B",[197,2061],{"x":2062,"y":2041,"width":2063,"height":2043,"rx":200,"fill":211,"stroke":212,"style":236},"370.5","82.7",[226,2065,2067],{"x":2066,"y":2050,"style":2051},"411.8","C",[197,2069],{"x":2070,"y":2041,"width":2071,"height":2043,"rx":200,"fill":211,"stroke":212,"style":236},"453.2","66.4",[226,2073,2075],{"x":2074,"y":2050,"style":2051},"486.4","D",[197,2077],{"x":2078,"y":2041,"width":2079,"height":2043,"rx":200,"fill":211,"stroke":212,"style":236},"519.6","62.9",[226,2081,2083],{"x":2082,"y":2050,"style":2051},"551.1","E",[197,2085],{"x":2019,"y":2086,"width":2042,"height":1237,"rx":200,"fill":211,"stroke":212,"style":236},"70.0",[226,2088,155],{"x":2089,"y":2090,"style":2051},"193.5","89.0",[197,2092],{"x":2036,"y":2086,"width":2046,"height":1237,"rx":200,"fill":2093,"stroke":2094,"style":1251},"#f0f4ff","var(--brand,#5b5cf0)",[226,2096,2098],{"x":2049,"y":2090,"style":2097},"font-size:11px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","column",[197,2100],{"x":2054,"y":2086,"width":2055,"height":1237,"rx":200,"fill":2093,"stroke":2094,"style":1251},[226,2102,2103],{"x":2058,"y":2090,"style":2097},"dtype",[197,2105],{"x":2062,"y":2086,"width":2063,"height":1237,"rx":200,"fill":2093,"stroke":2094,"style":1251},[226,2107,2108],{"x":2066,"y":2090,"style":2097},"missing %",[197,2110],{"x":2070,"y":2086,"width":2071,"height":1237,"rx":200,"fill":2093,"stroke":2094,"style":1251},[226,2112,2113],{"x":2074,"y":2090,"style":2097},"distinct",[197,2115],{"x":2078,"y":2086,"width":2079,"height":1237,"rx":200,"fill":2093,"stroke":2094,"style":1251},[226,2117,2118],{"x":2082,"y":2090,"style":2097},"unique",[197,2120],{"x":2019,"y":2121,"width":2042,"height":1237,"rx":200,"fill":211,"stroke":212,"style":236},"98.0",[226,2123,1843],{"x":2089,"y":2124,"style":2051},"117.0",[197,2126],{"x":2036,"y":2121,"width":2046,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2128,16],{"x":2049,"y":2124,"style":2129},"font-size:11px;font-weight:400;fill:var(--text,#172033);text-anchor:middle",[197,2131],{"x":2054,"y":2121,"width":2055,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2133,443],{"x":2058,"y":2124,"style":2129},[197,2135],{"x":2062,"y":2121,"width":2063,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2137,2138],{"x":2066,"y":2124,"style":2129},"0.0",[197,2140],{"x":2070,"y":2121,"width":2071,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2142,2143],{"x":2074,"y":2124,"style":2129},"499",[197,2145],{"x":2078,"y":2121,"width":2079,"height":1237,"rx":200,"fill":1249,"stroke":1250,"style":1251},[226,2147,2149],{"x":2082,"y":2124,"style":2148},"font-size:11px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","FALSE",[197,2151],{"x":2019,"y":2152,"width":2042,"height":1237,"rx":200,"fill":211,"stroke":212,"style":236},"126.0",[226,2154,415],{"x":2089,"y":2155,"style":2051},"145.0",[197,2157],{"x":2036,"y":2152,"width":2046,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2159,607],{"x":2049,"y":2155,"style":2129},[197,2161],{"x":2054,"y":2152,"width":2055,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2163,443],{"x":2058,"y":2155,"style":2129},[197,2165],{"x":2062,"y":2152,"width":2063,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2167,2138],{"x":2066,"y":2155,"style":2129},[197,2169],{"x":2070,"y":2152,"width":2071,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2171,199],{"x":2074,"y":2155,"style":2129},[197,2173],{"x":2078,"y":2152,"width":2079,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2175,2149],{"x":2082,"y":2155,"style":2129},[197,2177],{"x":2019,"y":2178,"width":2042,"height":1237,"rx":200,"fill":211,"stroke":212,"style":236},"154.0",[226,2180,199],{"x":2089,"y":2181,"style":2051},"173.0",[197,2183],{"x":2036,"y":2178,"width":2046,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2185,20],{"x":2049,"y":2181,"style":2129},[197,2187],{"x":2054,"y":2178,"width":2055,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2189,443],{"x":2058,"y":2181,"style":2129},[197,2191],{"x":2062,"y":2178,"width":2063,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2193,2025],{"x":2066,"y":2181,"style":2129},[197,2195],{"x":2070,"y":2178,"width":2071,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2197,2021],{"x":2074,"y":2181,"style":2129},[197,2199],{"x":2078,"y":2178,"width":2079,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2201,2149],{"x":2082,"y":2181,"style":2129},[197,2203],{"x":2019,"y":2204,"width":2042,"height":1237,"rx":200,"fill":211,"stroke":212,"style":236},"182.0",[226,2206,2207],{"x":2089,"y":209,"style":2051},"5",[197,2209],{"x":2036,"y":2204,"width":2046,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2211,167],{"x":2049,"y":209,"style":2129},[197,2213],{"x":2054,"y":2204,"width":2055,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2215,2216],{"x":2058,"y":209,"style":2129},"float64",[197,2218],{"x":2062,"y":2204,"width":2063,"height":1237,"rx":200,"fill":1276,"stroke":1277,"style":1251},[226,2220,2222],{"x":2066,"y":209,"style":2221},"font-size:11px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","58.0",[197,2224],{"x":2070,"y":2204,"width":2071,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2226,2227],{"x":2074,"y":209,"style":2129},"205",[197,2229],{"x":2078,"y":2204,"width":2079,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2231,2149],{"x":2082,"y":209,"style":2129},[197,2233],{"x":2019,"y":2234,"width":2042,"height":1237,"rx":200,"fill":211,"stroke":212,"style":236},"210.0",[226,2236,2238],{"x":2089,"y":2237,"style":2051},"229.0","6",[197,2240],{"x":2036,"y":2234,"width":2046,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2242,600],{"x":2049,"y":2237,"style":2129},[197,2244],{"x":2054,"y":2234,"width":2055,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2246,443],{"x":2058,"y":2237,"style":2129},[197,2248],{"x":2062,"y":2234,"width":2063,"height":1237,"rx":200,"fill":1276,"stroke":1277,"style":1251},[226,2250,2251],{"x":2066,"y":2237,"style":2221},"99.4",[197,2253],{"x":2070,"y":2234,"width":2071,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2255,1843],{"x":2074,"y":2237,"style":2129},[197,2257],{"x":2078,"y":2234,"width":2079,"height":1237,"rx":200,"fill":203,"stroke":212,"style":236},[226,2259,2149],{"x":2082,"y":2237,"style":2129},[38,2261,2263],{"id":2262},"common-pitfalls","Common pitfalls",[2265,2266,2267,2275,2281,2292,2298],"ul",{},[2268,2269,2270,2274],"li",{},[2271,2272,2273],"strong",{},"Profiling after cleaning."," Profile the raw export first; cleaning hides exactly the problems the profile should show.",[2268,2276,2277,2280],{},[2271,2278,2279],{},"Letting pandas guess types silently."," A text column that should be numeric is a finding; report it rather than coercing.",[2268,2282,2283,2291],{},[2271,2284,2285,2288,2289,1216],{},[14,2286,2287],{},"value_counts"," without ",[14,2290,404],{}," Decide whether blanks count as a value; report them separately as the missing share.",[2268,2293,2294,2297],{},[2271,2295,2296],{},"Treating unique columns as keys."," Uniqueness in one export is not a guarantee; check what the column means.",[2268,2299,2300,2303,2304,2307],{},[2271,2301,2302],{},"Profiling huge sheets row by row."," ",[14,2305,2306],{},"map"," over millions of values is slow; profile a sample for patterns and the full data for counts.",[38,2309,2311],{"id":2310},"conclusion","Conclusion",[10,2313,2314],{},"Before cleaning anything, build a one-row-per-column profile: type, missing share, distinct count and top values; the Python types mixed in each column; the shapes of text values; ranges and lengths; and whether each column is complete and unique. Write it to Excel with the worst columns highlighted, and use it to agree with the data's owner which problems are expected and which need fixing at the source.",[38,2316,2318],{"id":2317},"frequently-asked-questions","Frequently asked questions",[10,2320,2321,2324],{},[2271,2322,2323],{},"How do I profile a dataset in pandas?","\nLoop over the columns and collect the dtype, the share of missing values, the number of distinct values and the most common values for each, then add checks for mixed types, value patterns, ranges and uniqueness. Collect the results into a DataFrame with one row per column.",[10,2326,2327,2330],{},[2271,2328,2329],{},"Why doesn't df.describe() show my text columns?","\nBy default describe() summarises numeric columns only. describe(include=\"all\") adds count, unique, top and freq for text columns, but still says nothing about blanks, mixed types or patterns.",[10,2332,2333,2336],{},[2271,2334,2335],{},"How do I find a column that mixes numbers and text?","\nMap each non-missing value to its Python type name and count the types per column. A column that reports both float and str has numbers and text mixed, usually because of placeholders or formatted values.",[10,2338,2339,2342],{},[2271,2340,2341],{},"Are there libraries that do this automatically?","\nYes, packages such as ydata-profiling produce large HTML reports. A small profile you build yourself is quicker to run, easy to write to Excel and focused on the checks your data needs.",[38,2344,2346],{"id":2345},"related","Related",[2265,2348,2349,2355,2362,2369],{},[2268,2350,2351,2352,2354],{},"Up one level: ",[32,2353,35],{"href":34}," — checks that data is fit for use.",[2268,2356,2357,2361],{},[32,2358,2360],{"href":2359},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fcheck-excel-data-types-with-pandas\u002F","Check Excel Data Types with pandas"," — enforcing the types the profile reveals.",[2268,2363,2364,2368],{},[32,2365,2367],{"href":2366},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file\u002F","Find and Report Missing Values in an Excel File"," — blanks in more depth.",[2268,2370,2371,2375],{},[32,2372,2374],{"href":2373},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fstandardise-category-values-with-a-mapping-table\u002F","Standardise Category Values with a Mapping Table"," — fixing values like \"business\" and \"Business\".",[2377,2378,2379],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":48,"searchDepth":104,"depth":104,"links":2381},[2382,2383,2384,2385,2386,2387,2388,2389,2390,2391,2392],{"id":40,"depth":104,"text":41},{"id":335,"depth":104,"text":336},{"id":619,"depth":104,"text":620},{"id":865,"depth":104,"text":866},{"id":1304,"depth":104,"text":1305},{"id":1585,"depth":104,"text":1586},{"id":1714,"depth":104,"text":1715},{"id":2262,"depth":104,"text":2263},{"id":2310,"depth":104,"text":2311},{"id":2317,"depth":104,"text":2318},{"id":2345,"depth":104,"text":2346},"2026-09-29","Get to know a workbook before cleaning or loading it: one row per column with its type, blanks, distinct values, most common values, ranges and text lengths; columns that mix numbers and text; value patterns such as \"AA-9999\"; columns that could be a unique key; and the whole profile written to a workbook people can read.","md",[2397,2399,2401,2403],{"q":2323,"a":2398},"Loop over the columns and collect the dtype, the share of missing values, the number of distinct values and the most common values for each, then add checks for mixed types, value patterns, ranges and uniqueness. Collect the results into a DataFrame with one row per column.",{"q":2329,"a":2400},"By default describe() summarises numeric columns only. describe(include=\"all\") adds count, unique, top and freq for text columns, but still says nothing about blanks, mixed types or patterns.",{"q":2335,"a":2402},"Map each non-missing value to its Python type name and count the types per column. A column that reports both float and str has numbers and text mixed, usually because of placeholders or formatted values.",{"q":2341,"a":2404},"Yes, packages such as ydata-profiling produce large HTML reports. A small profile you build yourself is quicker to run, easy to write to Excel and focused on the checks your data needs.","profile an excel dataset with pandas",null,{"breadcrumb":2408},[2409,2412,2415],{"name":2410,"item":2411},"Home","\u002F",{"name":2413,"item":2414},"Advanced Data Transformation and Cleaning for Excel Automation","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":35,"item":34},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas",{"title":2418,"description":2419},"Profile an Excel File with pandas — Types, Blanks, Keys, Patterns","Build a data profile of an Excel sheet with pandas: per-column type, missing share, distinct count, top values, min\u002Fmax, mixed-type detection, value shape patterns, candidate keys and duplicates, written to an Excel profile report.","profile-an-excel-dataset-with-pandas","advanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas\u002Findex",[2423,2424],"data-validation","missing-data","how-to","T_eDEzPnBIQfnMfQfX6zkCv6vRlG7fH2Fy5QTMMMPCg",[2428,2432],{"title":2429,"path":2430,"stem":2431,"children":-1},"Highlight Invalid Cells in Excel with Python","\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fhighlight-invalid-cells-in-excel-with-python","advanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fhighlight-invalid-cells-in-excel-with-python\u002Findex",{"title":2433,"path":2434,"stem":2435,"children":-1},"Reconcile Excel Totals Against Control Totals with Python","\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Freconcile-excel-totals-with-control-totals","advanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Freconcile-excel-totals-with-control-totals\u002Findex",[2437,2439],{"path":2438,"title":2413},"\u002Fadvanced-data-transformation-and-cleaning",{"path":2440,"title":35},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python",[],[2443,2446,2449,2452,2455,2457],{"path":2444,"title":2445,"kind":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fdetect-outliers-in-excel-data-with-pandas","Detect Outliers in Excel Data with pandas",{"path":2447,"title":2448,"kind":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fapplying-conditional-formatting-with-openpyxl\u002Fhighlight-cells-containing-text-with-openpyxl","Highlight Cells Containing Specific Text with openpyxl",{"path":2450,"title":2451,"kind":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Ffind-near-duplicate-names-within-one-excel-column","Find Near-Duplicate Names Within One Excel Column",{"path":2453,"title":2454,"kind":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fread-dropdown-lists-from-an-existing-workbook","Read Dropdown Lists and Validation Rules from an Existing Workbook",{"path":2456,"title":2367,"kind":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file",{"path":2458,"title":2459,"kind":2425},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fflag-rows-with-too-many-missing-values","Flag Rows with Too Many Missing Values",1790972100740]