[{"data":1,"prerenderedAt":2668},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files":2634,"ancestors:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files":2643,"children:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files":2648,"related:\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files":2649},{"id":4,"title":5,"body":6,"dateModified":2601,"datePublished":2601,"description":2602,"extension":2603,"faq":2604,"keyword":2613,"kind":2614,"meta":2615,"navigation":108,"path":2624,"seo":2625,"slug":2628,"stem":2629,"topics":2630,"type":2632,"__hash__":2633},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files\u002Findex.md","Detect Schema Drift Between Monthly Excel Files",{"type":7,"value":8,"toc":2589},"minimark",[9,24,29,58,65,69,72,595,602,606,613,931,938,1041,1045,1048,1796,1803,1807,1810,2211,2218,2343,2467,2471,2478,2482,2517,2521,2524,2528,2534,2540,2546,2552,2556,2585],[10,11,12,13,17,18,23],"p",{},"The monthly export arrives, the report script runs, and nothing fails — but a column called \"Net amount\" became \"Net Amount (EUR)\" and the script quietly filled it with blanks, a new region \"Benelux\" appeared and fell outside every filter, and the quantity column now contains text because someone typed \"10 pcs\". This is ",[14,15,16],"em",{},"schema drift",": the structure or content of a recurring file changes without warning. Each change on its own is small; together they are the most common reason automated reports become silently wrong. The defence is cheap: describe each file's schema in a small snapshot — columns, types, blank rates, categories, ranges — compare it with the previous month's, and stop with a clear report when something important changed. This page builds that check with pandas. It belongs to ",[19,20,22],"a",{"href":21},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002F","Validating Excel Data with Python",".",[25,26,28],"h2",{"id":27},"prerequisites","Prerequisites",[30,31,36],"pre",{"className":32,"code":33,"language":34,"meta":35,"style":35},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash","",[37,38,39],"code",{"__ignoreMap":35},[40,41,44,48,52,55],"span",{"class":42,"line":43},"line",1,[40,45,47],{"class":46},"sMTad","pip",[40,49,51],{"class":50},"srMev"," install",[40,53,54],{"class":50}," pandas",[40,56,57],{"class":50}," openpyxl\n",[10,59,60,61,64],{},"The ",[37,62,63],{},"exports\u002F"," folder holds the August and September 2026 order exports from the same system. September's file has drifted in several ways at once — the kind of month every report owner eventually has. The task is to find out exactly how, automatically, before the September report is produced.",[25,66,68],{"id":67},"step-1-take-a-schema-snapshot","Step 1 — Take a schema snapshot",[10,70,71],{},"A snapshot is a small dictionary that describes a file well enough to compare months: each column's position, type, share of blanks, and either its distinct values (for categories) or its range (for numbers and dates):",[30,73,77],{"className":74,"code":75,"language":76,"meta":35,"style":35},"language-python shiki shiki-themes github-light github-dark-high-contrast","import json\nimport pandas as pd\n\ndef snapshot(path, max_categories=20):\n    df = pd.read_excel(path)\n    cols = {}\n    for pos, col in enumerate(df.columns):\n        s = df[col]\n        info = {\"position\": pos, \"dtype\": str(s.dtype), \"blank_share\": round(float(s.isna().mean()), 4)}\n        numeric = pd.to_numeric(s, errors=\"coerce\")\n        info[\"non_numeric\"] = int((numeric.isna() & s.notna()).sum()) if s.dtype == object else 0\n        if pd.api.types.is_numeric_dtype(s) or info[\"non_numeric\"] \u003C len(s) * 0.5 and s.dtype == object and numeric.notna().any():\n            info[\"min\"], info[\"max\"] = float(numeric.min()), float(numeric.max())\n        elif pd.api.types.is_datetime64_any_dtype(s):\n            info[\"min\"], info[\"max\"] = str(s.min().date()), str(s.max().date())\n        elif s.nunique() \u003C= max_categories:\n            info[\"categories\"] = sorted(map(str, s.dropna().unique()))\n        cols[col] = info\n    return {\"file\": str(path), \"rows\": len(df), \"columns\": cols}\n\naug = snapshot(\"exports\u002Forders_2026-08.xlsx\")\nsep = snapshot(\"exports\u002Forders_2026-09.xlsx\")\nprint(json.dumps(aug[\"columns\"][\"Region\"], indent=1))\nprint(json.dumps(sep[\"columns\"][\"Qty\"], indent=1))\n","python",[37,78,79,89,103,110,133,144,155,173,184,236,259,303,351,381,390,416,430,457,468,503,508,524,539,570],{"__ignoreMap":35},[40,80,81,85],{"class":42,"line":43},[40,82,84],{"class":83},"s-kum","import",[40,86,88],{"class":87},"skGVy"," json\n",[40,90,92,94,97,100],{"class":42,"line":91},2,[40,93,84],{"class":83},[40,95,96],{"class":87}," pandas ",[40,98,99],{"class":83},"as",[40,101,102],{"class":87}," pd\n",[40,104,106],{"class":42,"line":105},3,[40,107,109],{"emptyLinePlaceholder":108},true,"\n",[40,111,113,116,120,123,126,130],{"class":42,"line":112},4,[40,114,115],{"class":83},"def",[40,117,119],{"class":118},"s_Opv"," snapshot",[40,121,122],{"class":87},"(path, max_categories",[40,124,125],{"class":83},"=",[40,127,129],{"class":128},"sP0c6","20",[40,131,132],{"class":87},"):\n",[40,134,136,139,141],{"class":42,"line":135},5,[40,137,138],{"class":87},"    df ",[40,140,125],{"class":83},[40,142,143],{"class":87}," pd.read_excel(path)\n",[40,145,147,150,152],{"class":42,"line":146},6,[40,148,149],{"class":87},"    cols ",[40,151,125],{"class":83},[40,153,154],{"class":87}," {}\n",[40,156,158,161,164,167,170],{"class":42,"line":157},7,[40,159,160],{"class":83},"    for",[40,162,163],{"class":87}," pos, col ",[40,165,166],{"class":83},"in",[40,168,169],{"class":128}," enumerate",[40,171,172],{"class":87},"(df.columns):\n",[40,174,176,179,181],{"class":42,"line":175},8,[40,177,178],{"class":87},"        s ",[40,180,125],{"class":83},[40,182,183],{"class":87}," df[col]\n",[40,185,187,190,192,195,198,201,204,207,210,213,216,218,221,224,227,230,233],{"class":42,"line":186},9,[40,188,189],{"class":87},"        info ",[40,191,125],{"class":83},[40,193,194],{"class":87}," {",[40,196,197],{"class":50},"\"position\"",[40,199,200],{"class":87},": pos, ",[40,202,203],{"class":50},"\"dtype\"",[40,205,206],{"class":87},": ",[40,208,209],{"class":128},"str",[40,211,212],{"class":87},"(s.dtype), ",[40,214,215],{"class":50},"\"blank_share\"",[40,217,206],{"class":87},[40,219,220],{"class":128},"round",[40,222,223],{"class":87},"(",[40,225,226],{"class":128},"float",[40,228,229],{"class":87},"(s.isna().mean()), ",[40,231,232],{"class":128},"4",[40,234,235],{"class":87},")}\n",[40,237,239,242,244,247,251,253,256],{"class":42,"line":238},10,[40,240,241],{"class":87},"        numeric ",[40,243,125],{"class":83},[40,245,246],{"class":87}," pd.to_numeric(s, ",[40,248,250],{"class":249},"sa561","errors",[40,252,125],{"class":83},[40,254,255],{"class":50},"\"coerce\"",[40,257,258],{"class":87},")\n",[40,260,262,265,268,271,273,276,279,282,285,288,291,294,297,300],{"class":42,"line":261},11,[40,263,264],{"class":87},"        info[",[40,266,267],{"class":50},"\"non_numeric\"",[40,269,270],{"class":87},"] ",[40,272,125],{"class":83},[40,274,275],{"class":128}," int",[40,277,278],{"class":87},"((numeric.isna() ",[40,280,281],{"class":83},"&",[40,283,284],{"class":87}," s.notna()).sum()) ",[40,286,287],{"class":83},"if",[40,289,290],{"class":87}," s.dtype ",[40,292,293],{"class":83},"==",[40,295,296],{"class":128}," object",[40,298,299],{"class":83}," else",[40,301,302],{"class":128}," 0\n",[40,304,306,309,312,315,318,320,322,325,328,331,334,337,340,342,344,346,348],{"class":42,"line":305},12,[40,307,308],{"class":83},"        if",[40,310,311],{"class":87}," pd.api.types.is_numeric_dtype(s) ",[40,313,314],{"class":83},"or",[40,316,317],{"class":87}," info[",[40,319,267],{"class":50},[40,321,270],{"class":87},[40,323,324],{"class":83},"\u003C",[40,326,327],{"class":128}," len",[40,329,330],{"class":87},"(s) ",[40,332,333],{"class":83},"*",[40,335,336],{"class":128}," 0.5",[40,338,339],{"class":83}," and",[40,341,290],{"class":87},[40,343,293],{"class":83},[40,345,296],{"class":128},[40,347,339],{"class":83},[40,349,350],{"class":87}," numeric.notna().any():\n",[40,352,354,357,360,363,366,368,370,373,376,378],{"class":42,"line":353},13,[40,355,356],{"class":87},"            info[",[40,358,359],{"class":50},"\"min\"",[40,361,362],{"class":87},"], info[",[40,364,365],{"class":50},"\"max\"",[40,367,270],{"class":87},[40,369,125],{"class":83},[40,371,372],{"class":128}," float",[40,374,375],{"class":87},"(numeric.min()), ",[40,377,226],{"class":128},[40,379,380],{"class":87},"(numeric.max())\n",[40,382,384,387],{"class":42,"line":383},14,[40,385,386],{"class":83},"        elif",[40,388,389],{"class":87}," pd.api.types.is_datetime64_any_dtype(s):\n",[40,391,393,395,397,399,401,403,405,408,411,413],{"class":42,"line":392},15,[40,394,356],{"class":87},[40,396,359],{"class":50},[40,398,362],{"class":87},[40,400,365],{"class":50},[40,402,270],{"class":87},[40,404,125],{"class":83},[40,406,407],{"class":128}," str",[40,409,410],{"class":87},"(s.min().date()), ",[40,412,209],{"class":128},[40,414,415],{"class":87},"(s.max().date())\n",[40,417,419,421,424,427],{"class":42,"line":418},16,[40,420,386],{"class":83},[40,422,423],{"class":87}," s.nunique() ",[40,425,426],{"class":83},"\u003C=",[40,428,429],{"class":87}," max_categories:\n",[40,431,433,435,438,440,442,445,447,450,452,454],{"class":42,"line":432},17,[40,434,356],{"class":87},[40,436,437],{"class":50},"\"categories\"",[40,439,270],{"class":87},[40,441,125],{"class":83},[40,443,444],{"class":128}," sorted",[40,446,223],{"class":87},[40,448,449],{"class":128},"map",[40,451,223],{"class":87},[40,453,209],{"class":128},[40,455,456],{"class":87},", s.dropna().unique()))\n",[40,458,460,463,465],{"class":42,"line":459},18,[40,461,462],{"class":87},"        cols[col] ",[40,464,125],{"class":83},[40,466,467],{"class":87}," info\n",[40,469,471,474,476,479,481,483,486,489,491,494,497,500],{"class":42,"line":470},19,[40,472,473],{"class":83},"    return",[40,475,194],{"class":87},[40,477,478],{"class":50},"\"file\"",[40,480,206],{"class":87},[40,482,209],{"class":128},[40,484,485],{"class":87},"(path), ",[40,487,488],{"class":50},"\"rows\"",[40,490,206],{"class":87},[40,492,493],{"class":128},"len",[40,495,496],{"class":87},"(df), ",[40,498,499],{"class":50},"\"columns\"",[40,501,502],{"class":87},": cols}\n",[40,504,506],{"class":42,"line":505},20,[40,507,109],{"emptyLinePlaceholder":108},[40,509,511,514,516,519,522],{"class":42,"line":510},21,[40,512,513],{"class":87},"aug ",[40,515,125],{"class":83},[40,517,518],{"class":87}," snapshot(",[40,520,521],{"class":50},"\"exports\u002Forders_2026-08.xlsx\"",[40,523,258],{"class":87},[40,525,527,530,532,534,537],{"class":42,"line":526},22,[40,528,529],{"class":87},"sep ",[40,531,125],{"class":83},[40,533,518],{"class":87},[40,535,536],{"class":50},"\"exports\u002Forders_2026-09.xlsx\"",[40,538,258],{"class":87},[40,540,542,545,548,550,553,556,559,562,564,567],{"class":42,"line":541},23,[40,543,544],{"class":128},"print",[40,546,547],{"class":87},"(json.dumps(aug[",[40,549,499],{"class":50},[40,551,552],{"class":87},"][",[40,554,555],{"class":50},"\"Region\"",[40,557,558],{"class":87},"], ",[40,560,561],{"class":249},"indent",[40,563,125],{"class":83},[40,565,566],{"class":128},"1",[40,568,569],{"class":87},"))\n",[40,571,573,575,578,580,582,585,587,589,591,593],{"class":42,"line":572},24,[40,574,544],{"class":128},[40,576,577],{"class":87},"(json.dumps(sep[",[40,579,499],{"class":50},[40,581,552],{"class":87},[40,583,584],{"class":50},"\"Qty\"",[40,586,558],{"class":87},[40,588,561],{"class":249},[40,590,125],{"class":83},[40,592,566],{"class":128},[40,594,569],{"class":87},[10,596,597,598,601],{},"The snapshot records what matters for a report: Region's four categories and no blanks in August; Qty's type, range and — in September — two values that are not numbers. Categories are captured only for columns with few distinct values, so an ID column does not produce a huge list. Saving each month's snapshot as JSON (",[37,599,600],{},"json.dump(sep, open(...))",") gives a history of how the export has evolved, and means the comparison only needs the new file plus last month's small JSON, not last month's whole workbook.",[25,603,605],{"id":604},"step-2-compare-the-column-sets","Step 2 — Compare the column sets",[10,607,608,609,612],{},"The first comparison is structural: columns removed, added, or moved. Removed and added columns in the same month are often one ",[14,610,611],{},"renamed"," column, so suggest likely pairs by name similarity:",[30,614,616],{"className":74,"code":615,"language":76,"meta":35,"style":35},"from difflib import SequenceMatcher\n\nold_cols, new_cols = list(aug[\"columns\"]), list(sep[\"columns\"])\nremoved = [c for c in old_cols if c not in new_cols]\nadded = [c for c in new_cols if c not in old_cols]\nrenames = []\nfor r in removed:\n    best = max(added, key=lambda a: SequenceMatcher(None, r.lower(), a.lower()).ratio(), default=None)\n    if best and SequenceMatcher(None, r.lower(), best.lower()).ratio() >= 0.6:\n        renames.append((r, best, round(SequenceMatcher(None, r.lower(), best.lower()).ratio(), 2)))\nmoved = [c for c in old_cols if c in new_cols and aug[\"columns\"][c][\"position\"] != sep[\"columns\"][c][\"position\"]]\nprint(\"removed:\", removed, \"| added:\", added)\nprint(\"probable renames:\", renames)\nprint(\"moved:\", moved)\n",[37,617,618,631,635,664,698,727,737,749,786,814,835,889,907,919],{"__ignoreMap":35},[40,619,620,623,626,628],{"class":42,"line":43},[40,621,622],{"class":83},"from",[40,624,625],{"class":87}," difflib ",[40,627,84],{"class":83},[40,629,630],{"class":87}," SequenceMatcher\n",[40,632,633],{"class":42,"line":91},[40,634,109],{"emptyLinePlaceholder":108},[40,636,637,640,642,645,648,650,653,656,659,661],{"class":42,"line":105},[40,638,639],{"class":87},"old_cols, new_cols ",[40,641,125],{"class":83},[40,643,644],{"class":128}," list",[40,646,647],{"class":87},"(aug[",[40,649,499],{"class":50},[40,651,652],{"class":87},"]), ",[40,654,655],{"class":128},"list",[40,657,658],{"class":87},"(sep[",[40,660,499],{"class":50},[40,662,663],{"class":87},"])\n",[40,665,666,669,671,674,677,680,682,685,687,689,692,695],{"class":42,"line":112},[40,667,668],{"class":87},"removed ",[40,670,125],{"class":83},[40,672,673],{"class":87}," [c ",[40,675,676],{"class":83},"for",[40,678,679],{"class":87}," c ",[40,681,166],{"class":83},[40,683,684],{"class":87}," old_cols ",[40,686,287],{"class":83},[40,688,679],{"class":87},[40,690,691],{"class":83},"not",[40,693,694],{"class":83}," in",[40,696,697],{"class":87}," new_cols]\n",[40,699,700,703,705,707,709,711,713,716,718,720,722,724],{"class":42,"line":135},[40,701,702],{"class":87},"added ",[40,704,125],{"class":83},[40,706,673],{"class":87},[40,708,676],{"class":83},[40,710,679],{"class":87},[40,712,166],{"class":83},[40,714,715],{"class":87}," new_cols ",[40,717,287],{"class":83},[40,719,679],{"class":87},[40,721,691],{"class":83},[40,723,694],{"class":83},[40,725,726],{"class":87}," old_cols]\n",[40,728,729,732,734],{"class":42,"line":146},[40,730,731],{"class":87},"renames ",[40,733,125],{"class":83},[40,735,736],{"class":87}," []\n",[40,738,739,741,744,746],{"class":42,"line":157},[40,740,676],{"class":83},[40,742,743],{"class":87}," r ",[40,745,166],{"class":83},[40,747,748],{"class":87}," removed:\n",[40,750,751,754,756,759,762,765,768,771,774,777,780,782,784],{"class":42,"line":175},[40,752,753],{"class":87},"    best ",[40,755,125],{"class":83},[40,757,758],{"class":128}," max",[40,760,761],{"class":87},"(added, ",[40,763,764],{"class":249},"key",[40,766,767],{"class":83},"=lambda",[40,769,770],{"class":87}," a: SequenceMatcher(",[40,772,773],{"class":128},"None",[40,775,776],{"class":87},", r.lower(), a.lower()).ratio(), ",[40,778,779],{"class":249},"default",[40,781,125],{"class":83},[40,783,773],{"class":128},[40,785,258],{"class":87},[40,787,788,791,794,797,800,802,805,808,811],{"class":42,"line":186},[40,789,790],{"class":83},"    if",[40,792,793],{"class":87}," best ",[40,795,796],{"class":83},"and",[40,798,799],{"class":87}," SequenceMatcher(",[40,801,773],{"class":128},[40,803,804],{"class":87},", r.lower(), best.lower()).ratio() ",[40,806,807],{"class":83},">=",[40,809,810],{"class":128}," 0.6",[40,812,813],{"class":87},":\n",[40,815,816,819,821,824,826,829,832],{"class":42,"line":238},[40,817,818],{"class":87},"        renames.append((r, best, ",[40,820,220],{"class":128},[40,822,823],{"class":87},"(SequenceMatcher(",[40,825,773],{"class":128},[40,827,828],{"class":87},", r.lower(), best.lower()).ratio(), ",[40,830,831],{"class":128},"2",[40,833,834],{"class":87},")))\n",[40,836,837,840,842,844,846,848,850,852,854,856,858,860,862,865,867,870,872,874,877,880,882,884,886],{"class":42,"line":261},[40,838,839],{"class":87},"moved ",[40,841,125],{"class":83},[40,843,673],{"class":87},[40,845,676],{"class":83},[40,847,679],{"class":87},[40,849,166],{"class":83},[40,851,684],{"class":87},[40,853,287],{"class":83},[40,855,679],{"class":87},[40,857,166],{"class":83},[40,859,715],{"class":87},[40,861,796],{"class":83},[40,863,864],{"class":87}," aug[",[40,866,499],{"class":50},[40,868,869],{"class":87},"][c][",[40,871,197],{"class":50},[40,873,270],{"class":87},[40,875,876],{"class":83},"!=",[40,878,879],{"class":87}," sep[",[40,881,499],{"class":50},[40,883,869],{"class":87},[40,885,197],{"class":50},[40,887,888],{"class":87},"]]\n",[40,890,891,893,895,898,901,904],{"class":42,"line":305},[40,892,544],{"class":128},[40,894,223],{"class":87},[40,896,897],{"class":50},"\"removed:\"",[40,899,900],{"class":87},", removed, ",[40,902,903],{"class":50},"\"| added:\"",[40,905,906],{"class":87},", added)\n",[40,908,909,911,913,916],{"class":42,"line":353},[40,910,544],{"class":128},[40,912,223],{"class":87},[40,914,915],{"class":50},"\"probable renames:\"",[40,917,918],{"class":87},", renames)\n",[40,920,921,923,925,928],{"class":42,"line":383},[40,922,544],{"class":128},[40,924,223],{"class":87},[40,926,927],{"class":50},"\"moved:\"",[40,929,930],{"class":87},", moved)\n",[10,932,933,934,937],{},"\"Discount %\" was removed and \"Channel\" added, and \"Net amount\" is gone while \"Net Amount (EUR)\" appeared — ",[37,935,936],{},"SequenceMatcher"," scores that pair highly and suggests it as a rename, which is exactly what happened. A suggestion is not a decision: a person confirms the rename, and the report's column mapping is updated. Columns that merely moved — here Qty, pushed one place right by the new Channel column — matter only to code that reads columns by position, which is a good reason never to do that.",[939,940,948,949,948,953,948,957,948,964,948,974,948,985,948,992,948,998,948,1003,948,1007,948,1011,948,1016,948,1023,948,1028,948,1032,948,1035,948,1038],"svg",{"viewBox":941,"role":942,"ariaLabelledBy":943,"xmlns":946,"style":947},"4 0 752 207","img",[944,945],"sd-columns-t","sd-columns-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[950,951,952],"title",{"id":944},"How the column set changed between August and September",[954,955,956],"desc",{"id":945},"Discount percent was removed and Channel added. Net amount disappeared while Net Amount (EUR) appeared, which name similarity flags as a probable rename with a score of 0.77. Qty moved one position to the right.",[958,959],"rect",{"x":232,"y":960,"width":961,"height":962,"fill":963},"0","752","207","#ffffff",[958,965],{"x":966,"y":967,"width":968,"height":969,"rx":970,"fill":971,"stroke":972,"style":973},"20.0","16.0","351.0","175.0","14","#e7ebef","var(--line,#cdd5e6)","stroke-width:2px",[975,976,978],"g",{"transform":977},"translate(182.5,29.0) scale(1.083)",[979,980],"path",{"d":981,"fill":982,"stroke":983,"style":984},"M4 4h16v16H4z M4 9h16 M4 14h16 M10 4v16","none","var(--text,#172033)","stroke-width:1.66px;stroke-linecap:round;stroke-linejoin:round",[986,987,991],"text",{"x":988,"y":989,"style":990},"195.5","78.0","font-size:14px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","August",[42,993],{"x1":994,"y1":995,"x2":996,"y2":995,"stroke":972,"style":997},"36.0","91.0","355.0","stroke-width:1px",[986,999,1002],{"x":988,"y":1000,"style":1001},"111.0","font-size:12px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","Net amount",[986,1004,1006],{"x":988,"y":1005,"style":1001},"137.0","Discount %",[986,1008,1010],{"x":988,"y":1009,"style":1001},"163.0","Qty at position 3",[958,1012],{"x":1013,"y":967,"width":968,"height":969,"rx":970,"fill":1014,"stroke":1015,"style":973},"389.0","#fdefd8","var(--gold,#b4740a)",[975,1017,1019],{"transform":1018},"translate(551.5,29.0) scale(1.083)",[979,1020],{"d":1021,"fill":982,"stroke":1022,"style":984},"M12 3.5l9.5 17h-19z M12 10v5 M12 17.8v.4","var(--gold-ink,#7a4e06)",[986,1024,1027],{"x":1025,"y":989,"style":1026},"564.5","font-size:14px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","September",[42,1029],{"x1":1030,"y1":995,"x2":1031,"y2":995,"stroke":1015,"style":997},"405.0","724.0",[986,1033,1034],{"x":1025,"y":1000,"style":1001},"Net Amount (EUR) — probable rename",[986,1036,1037],{"x":1025,"y":1005,"style":1001},"Channel — new",[986,1039,1040],{"x":1025,"y":1009,"style":1001},"Qty at position 4",[25,1042,1044],{"id":1043},"step-3-compare-types-blanks-categories-and-ranges","Step 3 — Compare types, blanks, categories and ranges",[10,1046,1047],{},"For columns present in both months, compare each property of the snapshot and describe every change in words:",[30,1049,1051],{"className":74,"code":1050,"language":76,"meta":35,"style":35},"def column_drift(old, new, col):\n    issues = []\n    o, n = old[\"columns\"][col], new[\"columns\"][col]\n    if o[\"dtype\"] != n[\"dtype\"]:\n        issues.append(f\"type {o['dtype']} → {n['dtype']}\")\n    if n.get(\"non_numeric\", 0) > o.get(\"non_numeric\", 0):\n        issues.append(f\"{n['non_numeric']} non-numeric value(s)\")\n    if n[\"blank_share\"] - o[\"blank_share\"] > 0.05:\n        issues.append(f\"blanks {o['blank_share']:.0%} → {n['blank_share']:.0%}\")\n    if \"categories\" in o and \"categories\" in n:\n        new_vals = sorted(set(n[\"categories\"]) - set(o[\"categories\"]))\n        gone = sorted(set(o[\"categories\"]) - set(n[\"categories\"]))\n        if new_vals: issues.append(f\"new values {new_vals}\")\n        if gone: issues.append(f\"missing values {gone}\")\n    if \"min\" in o and \"min\" in n and isinstance(o[\"min\"], float):\n        span = (o[\"max\"] - o[\"min\"]) or 1\n        if abs(n[\"max\"] - o[\"max\"]) > 0.5 * span or abs(n[\"min\"] - o[\"min\"]) > 0.5 * span:\n            issues.append(f\"range {o['min']:g}–{o['max']:g} → {n['min']:g}–{n['max']:g}\")\n    return issues\n\ncommon = [c for c in old_cols if c in new_cols] + [(r, a) for r, a, _ in renames]\ndrift = {}\nfor c in common:\n    old_name, new_name = (c, c) if isinstance(c, str) else c\n    sep_view = {\"columns\": {old_name: sep[\"columns\"][new_name]}}\n    issues = column_drift(aug, sep_view, old_name)\n    if issues:\n        drift[new_name] = issues\nfor col, issues in drift.items():\n    print(f\"{col}: \" + \"; \".join(issues))\n",[37,1052,1053,1063,1072,1092,1113,1159,1190,1214,1240,1281,1302,1337,1366,1389,1412,1447,1474,1532,1604,1611,1615,1657,1666,1677,1704,1724,1734,1742,1752,1765],{"__ignoreMap":35},[40,1054,1055,1057,1060],{"class":42,"line":43},[40,1056,115],{"class":83},[40,1058,1059],{"class":118}," column_drift",[40,1061,1062],{"class":87},"(old, new, col):\n",[40,1064,1065,1068,1070],{"class":42,"line":91},[40,1066,1067],{"class":87},"    issues ",[40,1069,125],{"class":83},[40,1071,736],{"class":87},[40,1073,1074,1077,1079,1082,1084,1087,1089],{"class":42,"line":105},[40,1075,1076],{"class":87},"    o, n ",[40,1078,125],{"class":83},[40,1080,1081],{"class":87}," old[",[40,1083,499],{"class":50},[40,1085,1086],{"class":87},"][col], new[",[40,1088,499],{"class":50},[40,1090,1091],{"class":87},"][col]\n",[40,1093,1094,1096,1099,1101,1103,1105,1108,1110],{"class":42,"line":112},[40,1095,790],{"class":83},[40,1097,1098],{"class":87}," o[",[40,1100,203],{"class":50},[40,1102,270],{"class":87},[40,1104,876],{"class":83},[40,1106,1107],{"class":87}," n[",[40,1109,203],{"class":50},[40,1111,1112],{"class":87},"]:\n",[40,1114,1115,1118,1121,1124,1128,1131,1134,1137,1140,1143,1145,1148,1150,1152,1154,1157],{"class":42,"line":135},[40,1116,1117],{"class":87},"        issues.append(",[40,1119,1120],{"class":83},"f",[40,1122,1123],{"class":50},"\"type ",[40,1125,1127],{"class":1126},"sSjpA","{",[40,1129,1130],{"class":87},"o[",[40,1132,1133],{"class":50},"'dtype'",[40,1135,1136],{"class":87},"]",[40,1138,1139],{"class":1126},"}",[40,1141,1142],{"class":50}," → ",[40,1144,1127],{"class":1126},[40,1146,1147],{"class":87},"n[",[40,1149,1133],{"class":50},[40,1151,1136],{"class":87},[40,1153,1139],{"class":1126},[40,1155,1156],{"class":50},"\"",[40,1158,258],{"class":87},[40,1160,1161,1163,1166,1168,1171,1173,1176,1179,1182,1184,1186,1188],{"class":42,"line":146},[40,1162,790],{"class":83},[40,1164,1165],{"class":87}," n.get(",[40,1167,267],{"class":50},[40,1169,1170],{"class":87},", ",[40,1172,960],{"class":128},[40,1174,1175],{"class":87},") ",[40,1177,1178],{"class":83},">",[40,1180,1181],{"class":87}," o.get(",[40,1183,267],{"class":50},[40,1185,1170],{"class":87},[40,1187,960],{"class":128},[40,1189,132],{"class":87},[40,1191,1192,1194,1196,1198,1200,1202,1205,1207,1209,1212],{"class":42,"line":157},[40,1193,1117],{"class":87},[40,1195,1120],{"class":83},[40,1197,1156],{"class":50},[40,1199,1127],{"class":1126},[40,1201,1147],{"class":87},[40,1203,1204],{"class":50},"'non_numeric'",[40,1206,1136],{"class":87},[40,1208,1139],{"class":1126},[40,1210,1211],{"class":50}," non-numeric value(s)\"",[40,1213,258],{"class":87},[40,1215,1216,1218,1220,1222,1224,1227,1229,1231,1233,1235,1238],{"class":42,"line":175},[40,1217,790],{"class":83},[40,1219,1107],{"class":87},[40,1221,215],{"class":50},[40,1223,270],{"class":87},[40,1225,1226],{"class":83},"-",[40,1228,1098],{"class":87},[40,1230,215],{"class":50},[40,1232,270],{"class":87},[40,1234,1178],{"class":83},[40,1236,1237],{"class":128}," 0.05",[40,1239,813],{"class":87},[40,1241,1242,1244,1246,1249,1251,1253,1256,1258,1261,1263,1265,1267,1269,1271,1273,1275,1277,1279],{"class":42,"line":186},[40,1243,1117],{"class":87},[40,1245,1120],{"class":83},[40,1247,1248],{"class":50},"\"blanks ",[40,1250,1127],{"class":1126},[40,1252,1130],{"class":87},[40,1254,1255],{"class":50},"'blank_share'",[40,1257,1136],{"class":87},[40,1259,1260],{"class":83},":.0%",[40,1262,1139],{"class":1126},[40,1264,1142],{"class":50},[40,1266,1127],{"class":1126},[40,1268,1147],{"class":87},[40,1270,1255],{"class":50},[40,1272,1136],{"class":87},[40,1274,1260],{"class":83},[40,1276,1139],{"class":1126},[40,1278,1156],{"class":50},[40,1280,258],{"class":87},[40,1282,1283,1285,1288,1290,1293,1295,1297,1299],{"class":42,"line":238},[40,1284,790],{"class":83},[40,1286,1287],{"class":50}," \"categories\"",[40,1289,694],{"class":83},[40,1291,1292],{"class":87}," o ",[40,1294,796],{"class":83},[40,1296,1287],{"class":50},[40,1298,694],{"class":83},[40,1300,1301],{"class":87}," n:\n",[40,1303,1304,1307,1309,1311,1313,1316,1319,1321,1324,1326,1329,1332,1334],{"class":42,"line":261},[40,1305,1306],{"class":87},"        new_vals ",[40,1308,125],{"class":83},[40,1310,444],{"class":128},[40,1312,223],{"class":87},[40,1314,1315],{"class":128},"set",[40,1317,1318],{"class":87},"(n[",[40,1320,437],{"class":50},[40,1322,1323],{"class":87},"]) ",[40,1325,1226],{"class":83},[40,1327,1328],{"class":128}," set",[40,1330,1331],{"class":87},"(o[",[40,1333,437],{"class":50},[40,1335,1336],{"class":87},"]))\n",[40,1338,1339,1342,1344,1346,1348,1350,1352,1354,1356,1358,1360,1362,1364],{"class":42,"line":305},[40,1340,1341],{"class":87},"        gone ",[40,1343,125],{"class":83},[40,1345,444],{"class":128},[40,1347,223],{"class":87},[40,1349,1315],{"class":128},[40,1351,1331],{"class":87},[40,1353,437],{"class":50},[40,1355,1323],{"class":87},[40,1357,1226],{"class":83},[40,1359,1328],{"class":128},[40,1361,1318],{"class":87},[40,1363,437],{"class":50},[40,1365,1336],{"class":87},[40,1367,1368,1370,1373,1375,1378,1380,1383,1385,1387],{"class":42,"line":353},[40,1369,308],{"class":83},[40,1371,1372],{"class":87}," new_vals: issues.append(",[40,1374,1120],{"class":83},[40,1376,1377],{"class":50},"\"new values ",[40,1379,1127],{"class":1126},[40,1381,1382],{"class":87},"new_vals",[40,1384,1139],{"class":1126},[40,1386,1156],{"class":50},[40,1388,258],{"class":87},[40,1390,1391,1393,1396,1398,1401,1403,1406,1408,1410],{"class":42,"line":383},[40,1392,308],{"class":83},[40,1394,1395],{"class":87}," gone: issues.append(",[40,1397,1120],{"class":83},[40,1399,1400],{"class":50},"\"missing values ",[40,1402,1127],{"class":1126},[40,1404,1405],{"class":87},"gone",[40,1407,1139],{"class":1126},[40,1409,1156],{"class":50},[40,1411,258],{"class":87},[40,1413,1414,1416,1419,1421,1423,1425,1427,1429,1432,1434,1437,1439,1441,1443,1445],{"class":42,"line":392},[40,1415,790],{"class":83},[40,1417,1418],{"class":50}," \"min\"",[40,1420,694],{"class":83},[40,1422,1292],{"class":87},[40,1424,796],{"class":83},[40,1426,1418],{"class":50},[40,1428,694],{"class":83},[40,1430,1431],{"class":87}," n ",[40,1433,796],{"class":83},[40,1435,1436],{"class":128}," isinstance",[40,1438,1331],{"class":87},[40,1440,359],{"class":50},[40,1442,558],{"class":87},[40,1444,226],{"class":128},[40,1446,132],{"class":87},[40,1448,1449,1452,1454,1457,1459,1461,1463,1465,1467,1469,1471],{"class":42,"line":418},[40,1450,1451],{"class":87},"        span ",[40,1453,125],{"class":83},[40,1455,1456],{"class":87}," (o[",[40,1458,365],{"class":50},[40,1460,270],{"class":87},[40,1462,1226],{"class":83},[40,1464,1098],{"class":87},[40,1466,359],{"class":50},[40,1468,1323],{"class":87},[40,1470,314],{"class":83},[40,1472,1473],{"class":128}," 1\n",[40,1475,1476,1478,1481,1483,1485,1487,1489,1491,1493,1495,1497,1499,1502,1505,1507,1509,1511,1513,1515,1517,1519,1521,1523,1525,1527,1529],{"class":42,"line":432},[40,1477,308],{"class":83},[40,1479,1480],{"class":128}," abs",[40,1482,1318],{"class":87},[40,1484,365],{"class":50},[40,1486,270],{"class":87},[40,1488,1226],{"class":83},[40,1490,1098],{"class":87},[40,1492,365],{"class":50},[40,1494,1323],{"class":87},[40,1496,1178],{"class":83},[40,1498,336],{"class":128},[40,1500,1501],{"class":83}," *",[40,1503,1504],{"class":87}," span ",[40,1506,314],{"class":83},[40,1508,1480],{"class":128},[40,1510,1318],{"class":87},[40,1512,359],{"class":50},[40,1514,270],{"class":87},[40,1516,1226],{"class":83},[40,1518,1098],{"class":87},[40,1520,359],{"class":50},[40,1522,1323],{"class":87},[40,1524,1178],{"class":83},[40,1526,336],{"class":128},[40,1528,1501],{"class":83},[40,1530,1531],{"class":87}," span:\n",[40,1533,1534,1537,1539,1542,1544,1546,1549,1551,1554,1556,1559,1561,1563,1566,1568,1570,1572,1574,1576,1578,1580,1582,1584,1586,1588,1590,1592,1594,1596,1598,1600,1602],{"class":42,"line":459},[40,1535,1536],{"class":87},"            issues.append(",[40,1538,1120],{"class":83},[40,1540,1541],{"class":50},"\"range ",[40,1543,1127],{"class":1126},[40,1545,1130],{"class":87},[40,1547,1548],{"class":50},"'min'",[40,1550,1136],{"class":87},[40,1552,1553],{"class":83},":g",[40,1555,1139],{"class":1126},[40,1557,1558],{"class":50},"–",[40,1560,1127],{"class":1126},[40,1562,1130],{"class":87},[40,1564,1565],{"class":50},"'max'",[40,1567,1136],{"class":87},[40,1569,1553],{"class":83},[40,1571,1139],{"class":1126},[40,1573,1142],{"class":50},[40,1575,1127],{"class":1126},[40,1577,1147],{"class":87},[40,1579,1548],{"class":50},[40,1581,1136],{"class":87},[40,1583,1553],{"class":83},[40,1585,1139],{"class":1126},[40,1587,1558],{"class":50},[40,1589,1127],{"class":1126},[40,1591,1147],{"class":87},[40,1593,1565],{"class":50},[40,1595,1136],{"class":87},[40,1597,1553],{"class":83},[40,1599,1139],{"class":1126},[40,1601,1156],{"class":50},[40,1603,258],{"class":87},[40,1605,1606,1608],{"class":42,"line":470},[40,1607,473],{"class":83},[40,1609,1610],{"class":87}," issues\n",[40,1612,1613],{"class":42,"line":505},[40,1614,109],{"emptyLinePlaceholder":108},[40,1616,1617,1620,1622,1624,1626,1628,1630,1632,1634,1636,1638,1641,1644,1647,1649,1652,1654],{"class":42,"line":510},[40,1618,1619],{"class":87},"common ",[40,1621,125],{"class":83},[40,1623,673],{"class":87},[40,1625,676],{"class":83},[40,1627,679],{"class":87},[40,1629,166],{"class":83},[40,1631,684],{"class":87},[40,1633,287],{"class":83},[40,1635,679],{"class":87},[40,1637,166],{"class":83},[40,1639,1640],{"class":87}," new_cols] ",[40,1642,1643],{"class":83},"+",[40,1645,1646],{"class":87}," [(r, a) ",[40,1648,676],{"class":83},[40,1650,1651],{"class":87}," r, a, _ ",[40,1653,166],{"class":83},[40,1655,1656],{"class":87}," renames]\n",[40,1658,1659,1662,1664],{"class":42,"line":526},[40,1660,1661],{"class":87},"drift ",[40,1663,125],{"class":83},[40,1665,154],{"class":87},[40,1667,1668,1670,1672,1674],{"class":42,"line":541},[40,1669,676],{"class":83},[40,1671,679],{"class":87},[40,1673,166],{"class":83},[40,1675,1676],{"class":87}," common:\n",[40,1678,1679,1682,1684,1687,1689,1691,1694,1696,1698,1701],{"class":42,"line":572},[40,1680,1681],{"class":87},"    old_name, new_name ",[40,1683,125],{"class":83},[40,1685,1686],{"class":87}," (c, c) ",[40,1688,287],{"class":83},[40,1690,1436],{"class":128},[40,1692,1693],{"class":87},"(c, ",[40,1695,209],{"class":128},[40,1697,1175],{"class":87},[40,1699,1700],{"class":83},"else",[40,1702,1703],{"class":87}," c\n",[40,1705,1707,1710,1712,1714,1716,1719,1721],{"class":42,"line":1706},25,[40,1708,1709],{"class":87},"    sep_view ",[40,1711,125],{"class":83},[40,1713,194],{"class":87},[40,1715,499],{"class":50},[40,1717,1718],{"class":87},": {old_name: sep[",[40,1720,499],{"class":50},[40,1722,1723],{"class":87},"][new_name]}}\n",[40,1725,1727,1729,1731],{"class":42,"line":1726},26,[40,1728,1067],{"class":87},[40,1730,125],{"class":83},[40,1732,1733],{"class":87}," column_drift(aug, sep_view, old_name)\n",[40,1735,1737,1739],{"class":42,"line":1736},27,[40,1738,790],{"class":83},[40,1740,1741],{"class":87}," issues:\n",[40,1743,1745,1748,1750],{"class":42,"line":1744},28,[40,1746,1747],{"class":87},"        drift[new_name] ",[40,1749,125],{"class":83},[40,1751,1610],{"class":87},[40,1753,1755,1757,1760,1762],{"class":42,"line":1754},29,[40,1756,676],{"class":83},[40,1758,1759],{"class":87}," col, issues ",[40,1761,166],{"class":83},[40,1763,1764],{"class":87}," drift.items():\n",[40,1766,1768,1771,1773,1775,1777,1779,1782,1784,1787,1790,1793],{"class":42,"line":1767},30,[40,1769,1770],{"class":128},"    print",[40,1772,223],{"class":87},[40,1774,1120],{"class":83},[40,1776,1156],{"class":50},[40,1778,1127],{"class":1126},[40,1780,1781],{"class":87},"col",[40,1783,1139],{"class":1126},[40,1785,1786],{"class":50},": \"",[40,1788,1789],{"class":83}," +",[40,1791,1792],{"class":50}," \"; \"",[40,1794,1795],{"class":87},".join(issues))\n",[10,1797,1798,1799,1802],{},"Region has a new value, \"Benelux\", and its blank share jumped from 0% to 15% — both reasons a regional report would quietly go wrong. Qty contains two non-numeric values, the \"10 pcs\" and \"approx 4\" that would break any sum, and its type changed from integer to ",[37,1800,1801],{},"object"," as a result. The renamed amount column is compared under its old name, so its type and range are checked too. Thresholds — a 5-point rise in blanks, a range moving by half its previous span — are deliberately simple; tune them to how much each column normally varies. The comparisons stay explainable, which matters when the output is a message to the team that owns the export.",[25,1804,1806],{"id":1805},"step-4-a-drift-report-and-a-decision","Step 4 — A drift report and a decision",[10,1808,1809],{},"Gather everything into one report, classify each finding as blocking or informational, and stop the pipeline when anything blocking is present:",[30,1811,1813],{"className":74,"code":1812,"language":76,"meta":35,"style":35},"REQUIRED = {\"Order ID\", \"Order date\", \"Region\", \"Qty\", \"Net amount\"}\nfindings = []\nfor c in removed:\n    renamed_to = next((a for r, a, _ in renames if r == c), None)\n    level = \"warn\" if renamed_to else (\"block\" if c in REQUIRED else \"warn\")\n    findings.append((level, c, f\"removed\" + (f\" — probably renamed to '{renamed_to}'\" if renamed_to else \"\")))\nfor c in added:\n    if c not in [a for _, a, _ in renames]:\n        findings.append((\"info\", c, \"new column\"))\nfor col, issues in drift.items():\n    for i in issues:\n        findings.append((\"block\" if \"non-numeric\" in i or \"type\" in i else \"warn\", col, i))\n\nreport = pd.DataFrame(findings, columns=[\"Level\", \"Column\", \"Change\"]).sort_values(\"Level\")\nprint(report.to_string(index=False))\nblocking = report[report[\"Level\"] == \"block\"]\nprint(\"RESULT:\", \"stop — fix or confirm before running the report\" if len(blocking) else \"proceed\")\nreport.to_excel(\"drift_report_2026-09.xlsx\", index=False)\n",[37,1814,1815,1849,1858,1868,1903,1942,1982,1993,2016,2032,2042,2053,2084,2088,2126,2143,2165,2193],{"__ignoreMap":35},[40,1816,1817,1820,1823,1825,1828,1830,1833,1835,1837,1839,1841,1843,1846],{"class":42,"line":43},[40,1818,1819],{"class":128},"REQUIRED",[40,1821,1822],{"class":83}," =",[40,1824,194],{"class":87},[40,1826,1827],{"class":50},"\"Order ID\"",[40,1829,1170],{"class":87},[40,1831,1832],{"class":50},"\"Order date\"",[40,1834,1170],{"class":87},[40,1836,555],{"class":50},[40,1838,1170],{"class":87},[40,1840,584],{"class":50},[40,1842,1170],{"class":87},[40,1844,1845],{"class":50},"\"Net amount\"",[40,1847,1848],{"class":87},"}\n",[40,1850,1851,1854,1856],{"class":42,"line":91},[40,1852,1853],{"class":87},"findings ",[40,1855,125],{"class":83},[40,1857,736],{"class":87},[40,1859,1860,1862,1864,1866],{"class":42,"line":105},[40,1861,676],{"class":83},[40,1863,679],{"class":87},[40,1865,166],{"class":83},[40,1867,748],{"class":87},[40,1869,1870,1873,1875,1878,1881,1883,1885,1887,1890,1892,1894,1896,1899,1901],{"class":42,"line":112},[40,1871,1872],{"class":87},"    renamed_to ",[40,1874,125],{"class":83},[40,1876,1877],{"class":128}," next",[40,1879,1880],{"class":87},"((a ",[40,1882,676],{"class":83},[40,1884,1651],{"class":87},[40,1886,166],{"class":83},[40,1888,1889],{"class":87}," renames ",[40,1891,287],{"class":83},[40,1893,743],{"class":87},[40,1895,293],{"class":83},[40,1897,1898],{"class":87}," c), ",[40,1900,773],{"class":128},[40,1902,258],{"class":87},[40,1904,1905,1908,1910,1913,1916,1919,1921,1924,1927,1929,1931,1933,1936,1938,1940],{"class":42,"line":135},[40,1906,1907],{"class":87},"    level ",[40,1909,125],{"class":83},[40,1911,1912],{"class":50}," \"warn\"",[40,1914,1915],{"class":83}," if",[40,1917,1918],{"class":87}," renamed_to ",[40,1920,1700],{"class":83},[40,1922,1923],{"class":87}," (",[40,1925,1926],{"class":50},"\"block\"",[40,1928,1915],{"class":83},[40,1930,679],{"class":87},[40,1932,166],{"class":83},[40,1934,1935],{"class":128}," REQUIRED",[40,1937,299],{"class":83},[40,1939,1912],{"class":50},[40,1941,258],{"class":87},[40,1943,1944,1947,1949,1952,1954,1956,1958,1961,1963,1966,1968,1971,1973,1975,1977,1980],{"class":42,"line":146},[40,1945,1946],{"class":87},"    findings.append((level, c, ",[40,1948,1120],{"class":83},[40,1950,1951],{"class":50},"\"removed\"",[40,1953,1789],{"class":83},[40,1955,1923],{"class":87},[40,1957,1120],{"class":83},[40,1959,1960],{"class":50},"\" — probably renamed to '",[40,1962,1127],{"class":1126},[40,1964,1965],{"class":87},"renamed_to",[40,1967,1139],{"class":1126},[40,1969,1970],{"class":50},"'\"",[40,1972,1915],{"class":83},[40,1974,1918],{"class":87},[40,1976,1700],{"class":83},[40,1978,1979],{"class":50}," \"\"",[40,1981,834],{"class":87},[40,1983,1984,1986,1988,1990],{"class":42,"line":157},[40,1985,676],{"class":83},[40,1987,679],{"class":87},[40,1989,166],{"class":83},[40,1991,1992],{"class":87}," added:\n",[40,1994,1995,1997,1999,2001,2003,2006,2008,2011,2013],{"class":42,"line":175},[40,1996,790],{"class":83},[40,1998,679],{"class":87},[40,2000,691],{"class":83},[40,2002,694],{"class":83},[40,2004,2005],{"class":87}," [a ",[40,2007,676],{"class":83},[40,2009,2010],{"class":87}," _, a, _ ",[40,2012,166],{"class":83},[40,2014,2015],{"class":87}," renames]:\n",[40,2017,2018,2021,2024,2027,2030],{"class":42,"line":186},[40,2019,2020],{"class":87},"        findings.append((",[40,2022,2023],{"class":50},"\"info\"",[40,2025,2026],{"class":87},", c, ",[40,2028,2029],{"class":50},"\"new column\"",[40,2031,569],{"class":87},[40,2033,2034,2036,2038,2040],{"class":42,"line":238},[40,2035,676],{"class":83},[40,2037,1759],{"class":87},[40,2039,166],{"class":83},[40,2041,1764],{"class":87},[40,2043,2044,2046,2049,2051],{"class":42,"line":261},[40,2045,160],{"class":83},[40,2047,2048],{"class":87}," i ",[40,2050,166],{"class":83},[40,2052,1741],{"class":87},[40,2054,2055,2057,2059,2061,2064,2066,2068,2070,2073,2075,2077,2079,2081],{"class":42,"line":305},[40,2056,2020],{"class":87},[40,2058,1926],{"class":50},[40,2060,1915],{"class":83},[40,2062,2063],{"class":50}," \"non-numeric\"",[40,2065,694],{"class":83},[40,2067,2048],{"class":87},[40,2069,314],{"class":83},[40,2071,2072],{"class":50}," \"type\"",[40,2074,694],{"class":83},[40,2076,2048],{"class":87},[40,2078,1700],{"class":83},[40,2080,1912],{"class":50},[40,2082,2083],{"class":87},", col, i))\n",[40,2085,2086],{"class":42,"line":353},[40,2087,109],{"emptyLinePlaceholder":108},[40,2089,2090,2093,2095,2098,2101,2103,2106,2109,2111,2114,2116,2119,2122,2124],{"class":42,"line":383},[40,2091,2092],{"class":87},"report ",[40,2094,125],{"class":83},[40,2096,2097],{"class":87}," pd.DataFrame(findings, ",[40,2099,2100],{"class":249},"columns",[40,2102,125],{"class":83},[40,2104,2105],{"class":87},"[",[40,2107,2108],{"class":50},"\"Level\"",[40,2110,1170],{"class":87},[40,2112,2113],{"class":50},"\"Column\"",[40,2115,1170],{"class":87},[40,2117,2118],{"class":50},"\"Change\"",[40,2120,2121],{"class":87},"]).sort_values(",[40,2123,2108],{"class":50},[40,2125,258],{"class":87},[40,2127,2128,2130,2133,2136,2138,2141],{"class":42,"line":392},[40,2129,544],{"class":128},[40,2131,2132],{"class":87},"(report.to_string(",[40,2134,2135],{"class":249},"index",[40,2137,125],{"class":83},[40,2139,2140],{"class":128},"False",[40,2142,569],{"class":87},[40,2144,2145,2148,2150,2153,2155,2157,2159,2162],{"class":42,"line":418},[40,2146,2147],{"class":87},"blocking ",[40,2149,125],{"class":83},[40,2151,2152],{"class":87}," report[report[",[40,2154,2108],{"class":50},[40,2156,270],{"class":87},[40,2158,293],{"class":83},[40,2160,2161],{"class":50}," \"block\"",[40,2163,2164],{"class":87},"]\n",[40,2166,2167,2169,2171,2174,2176,2179,2181,2183,2186,2188,2191],{"class":42,"line":432},[40,2168,544],{"class":128},[40,2170,223],{"class":87},[40,2172,2173],{"class":50},"\"RESULT:\"",[40,2175,1170],{"class":87},[40,2177,2178],{"class":50},"\"stop — fix or confirm before running the report\"",[40,2180,1915],{"class":83},[40,2182,327],{"class":128},[40,2184,2185],{"class":87},"(blocking) ",[40,2187,1700],{"class":83},[40,2189,2190],{"class":50}," \"proceed\"",[40,2192,258],{"class":87},[40,2194,2195,2198,2201,2203,2205,2207,2209],{"class":42,"line":459},[40,2196,2197],{"class":87},"report.to_excel(",[40,2199,2200],{"class":50},"\"drift_report_2026-09.xlsx\"",[40,2202,1170],{"class":87},[40,2204,2135],{"class":249},[40,2206,125],{"class":83},[40,2208,2140],{"class":128},[40,2210,258],{"class":87},[10,2212,2213,2214,23],{},"The report lists every change with a level. Non-numeric quantities are blocking, because sums would fail or be wrong; the rename is a warning, because the mapping must be confirmed but the data is there; a new column is information. Here the result is \"stop\": the two text quantities must be fixed or explicitly handled before the September report runs. Writing the report to a workbook — or sending it to the export's owners — turns a silent break into a conversation on the day the file arrives. Converting confirmed changes into an updated reading contract is covered in ",[19,2215,2217],{"href":2216},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fvalidate-excel-columns-before-import-with-pandas\u002F","Validate Excel Columns Before Import with pandas",[939,2219,948,2224,948,2227,948,2230,948,2233,948,2238,948,2245,948,2251,948,2256,948,2260,948,2266,948,2271,948,2275,948,2282,948,2287,948,2290,948,2292,948,2295,948,2298,948,2301,948,2306,948,2312,948,2316,948,2320,948,2323,948,2328,948,2335,948,2340],{"viewBox":2220,"role":942,"ariaLabelledBy":2221,"xmlns":946,"style":947},"4 0 752 142",[2222,2223],"sd-flow-t","sd-flow-d",[950,2225,2226],{"id":2222},"A drift check on every new file",[954,2228,2229],{"id":2223},"Each new file is summarised in a schema snapshot, compared with last month's snapshot for column changes and per-column drift, the findings are classified as blocking, warning or information, and the pipeline stops on any blocking finding while the snapshot is saved for next month.",[958,2231],{"x":232,"y":960,"width":961,"height":2232,"fill":963},"142",[958,2234],{"x":966,"y":967,"width":2235,"height":2236,"rx":2237,"fill":971,"stroke":972,"style":973},"153.0","110.5","12",[975,2239,2241],{"transform":2240},"translate(84.5,29.0) scale(1.000)",[979,2242],{"d":2243,"fill":982,"stroke":983,"style":2244},"M11 4a7 7 0 1 0 .01 0z M16 16l5 5","stroke-width:1.80px;stroke-linecap:round;stroke-linejoin:round",[986,2246,2250],{"x":2247,"y":2248,"style":2249},"96.5","75.5","font-size:13.5px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","Snapshot",[986,2252,2255],{"x":2247,"y":2253,"style":2254},"95.0","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","columns, types,",[986,2257,2259],{"x":2247,"y":2258,"style":2254},"112.5","categories",[42,2261],{"x1":2262,"y1":2263,"x2":2264,"y2":2263,"stroke":2265,"style":973},"178.0","71.2","200.0","var(--brand,#5b5cf0)",[2267,2268],"polygon",{"points":2269,"fill":2270},"204.0,71.2 195.0,76.2 195.0,66.2","#5b5cf0",[958,2272],{"x":2273,"y":967,"width":2235,"height":2236,"rx":2237,"fill":2274,"stroke":2265,"style":973},"209.0","#f0f4ff",[975,2276,2278],{"transform":2277},"translate(273.5,29.0) scale(1.000)",[979,2279],{"d":2280,"fill":982,"stroke":2281,"style":2244},"M12 3l9 5-9 5-9-5z M3 13l9 5 9-5 M3 17l9 5 9-5","var(--brand-strong,#4338ca)",[986,2283,2286],{"x":2284,"y":2248,"style":2285},"285.5","font-size:13.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Compare",[986,2288,2289],{"x":2284,"y":2253,"style":2254},"added, removed,",[986,2291,611],{"x":2284,"y":2258,"style":2254},[42,2293],{"x1":2294,"y1":2263,"x2":1013,"y2":2263,"stroke":2265,"style":973},"367.0",[2267,2296],{"points":2297,"fill":2270},"393.0,71.2 384.0,76.2 384.0,66.2",[958,2299],{"x":2300,"y":967,"width":2235,"height":2236,"rx":2237,"fill":1014,"stroke":1015,"style":973},"398.0",[975,2302,2304],{"transform":2303},"translate(462.5,37.8) scale(1.000)",[979,2305],{"d":1021,"fill":982,"stroke":1022,"style":2244},[986,2307,2311],{"x":2308,"y":2309,"style":2310},"474.5","84.2","font-size:13.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","Classify",[986,2313,2315],{"x":2308,"y":2314,"style":2254},"103.8","block · warn · info",[42,2317],{"x1":2318,"y1":2263,"x2":2319,"y2":2263,"stroke":2265,"style":973},"556.0","578.0",[2267,2321],{"points":2322,"fill":2270},"582.0,71.2 573.0,76.2 573.0,66.2",[958,2324],{"x":2325,"y":967,"width":2235,"height":2236,"rx":2237,"fill":2326,"stroke":2327,"style":973},"587.0","#d9f4f1","var(--teal,#0f9488)",[975,2329,2331],{"transform":2330},"translate(651.5,37.8) scale(1.000)",[979,2332],{"d":2333,"fill":982,"stroke":2334,"style":2244},"M5 12.5l4.5 4.5L19 7","var(--teal-ink,#0b6157)",[986,2336,2339],{"x":2337,"y":2309,"style":2338},"663.5","font-size:13.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","Decide",[986,2341,2342],{"x":2337,"y":2314,"style":2254},"stop or proceed",[939,2344,948,2349,948,2352,948,2355,948,2358,948,2362,948,2367,948,2373,948,2376,948,2380,948,2384,948,2389,948,2391,948,2395,948,2397,948,2400,948,2403,948,2406,948,2408,948,2411,948,2413,948,2416,948,2419,948,2422,948,2424,948,2428,948,2430,948,2432,948,2435,948,2439,948,2441,948,2445,948,2447,948,2449,948,2452,948,2456,948,2458,948,2462,948,2464],{"viewBox":2345,"role":942,"ariaLabelledBy":2346,"xmlns":946,"style":947},"4 0 752 242",[2347,2348],"sd-findings-t","sd-findings-d",[950,2350,2351],{"id":2347},"The September drift report",[954,2353,2354],{"id":2348},"Qty is blocking because of a type change and two non-numeric values. Net amount was probably renamed, Discount percent removed, and Region gained a new value and many blanks, all warnings. Channel is a new column, for information.",[958,2356],{"x":232,"y":960,"width":961,"height":2357,"fill":963},"242",[958,2359],{"x":966,"y":967,"width":2360,"height":2361,"rx":960,"fill":963,"stroke":972,"style":997},"102.5","40.0",[958,2363],{"x":2364,"y":967,"width":2365,"height":2361,"rx":960,"fill":2274,"stroke":2265,"style":2366},"122.5","308.7","stroke-width:1.2px",[986,2368,2372],{"x":2369,"y":2370,"style":2371},"276.9","41.0","font-size:13px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","change",[958,2374],{"x":2375,"y":967,"width":2365,"height":2361,"rx":960,"fill":2274,"stroke":2265,"style":2366},"431.3",[986,2377,2379],{"x":2378,"y":2370,"style":2371},"585.6","level",[958,2381],{"x":966,"y":2382,"width":2360,"height":2383,"rx":960,"fill":971,"stroke":972,"style":997},"56.0","34.0",[986,2385,2388],{"x":2386,"y":989,"style":2387},"32.0","font-size:12.5px;font-weight:700;fill:var(--text,#172033);text-anchor:start","Qty",[958,2390],{"x":2364,"y":2382,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2392,2394],{"x":2369,"y":2393,"style":2254},"77.0","int64 → object, 2 text values",[958,2396],{"x":2375,"y":2382,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2398,2399],{"x":2378,"y":2393,"style":2254},"block",[958,2401],{"x":966,"y":2402,"width":2360,"height":2383,"rx":960,"fill":971,"stroke":972,"style":997},"90.0",[986,2404,1002],{"x":2386,"y":2405,"style":2387},"112.0",[958,2407],{"x":2364,"y":2402,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2409,2410],{"x":2369,"y":1000,"style":2254},"→ Net Amount (EUR)?",[958,2412],{"x":2375,"y":2402,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2414,2415],{"x":2378,"y":1000,"style":2254},"warn",[958,2417],{"x":966,"y":2418,"width":2360,"height":2383,"rx":960,"fill":971,"stroke":972,"style":997},"124.0",[986,2420,1006],{"x":2386,"y":2421,"style":2387},"146.0",[958,2423],{"x":2364,"y":2418,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2425,2427],{"x":2369,"y":2426,"style":2254},"145.0","removed",[958,2429],{"x":2375,"y":2418,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2431,2415],{"x":2378,"y":2426,"style":2254},[958,2433],{"x":966,"y":2434,"width":2360,"height":2383,"rx":960,"fill":971,"stroke":972,"style":997},"158.0",[986,2436,2438],{"x":2386,"y":2437,"style":2387},"180.0","Region",[958,2440],{"x":2364,"y":2434,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2442,2444],{"x":2369,"y":2443,"style":2254},"179.0","Benelux new, blanks 0% → 15%",[958,2446],{"x":2375,"y":2434,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2448,2415],{"x":2378,"y":2443,"style":2254},[958,2450],{"x":966,"y":2451,"width":2360,"height":2383,"rx":960,"fill":971,"stroke":972,"style":997},"192.0",[986,2453,2455],{"x":2386,"y":2454,"style":2387},"214.0","Channel",[958,2457],{"x":2364,"y":2451,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2459,2461],{"x":2369,"y":2460,"style":2254},"213.0","new column",[958,2463],{"x":2375,"y":2451,"width":2365,"height":2383,"rx":960,"fill":963,"stroke":972,"style":997},[986,2465,2466],{"x":2378,"y":2460,"style":2254},"info",[25,2468,2470],{"id":2469},"making-it-routine","Making it routine",[10,2472,2473,2474,23],{},"Run the snapshot and comparison as the first step of every scheduled report, store each month's snapshot alongside the file, and keep the thresholds and required columns in a configuration file rather than in code. Over a few months the snapshots also become documentation — when did the Channel column appear? when did blanks in Region start? — that nobody had to write by hand. For row-level rules on top of these file-level checks, schema libraries such as pandera are a good fit; see ",[19,2475,2477],{"href":2476},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fvalidate-excel-data-with-pandera-schemas\u002F","Validate Excel Data with pandera Schemas",[25,2479,2481],{"id":2480},"common-pitfalls","Common pitfalls",[2483,2484,2485,2493,2499,2505,2511],"ul",{},[2486,2487,2488,2492],"li",{},[2489,2490,2491],"strong",{},"Checking only that the script ran."," Drift rarely raises an error; compare structure and content.",[2486,2494,2495,2498],{},[2489,2496,2497],{},"Reading columns by position."," A new column shifts everything; read by name.",[2486,2500,2501,2504],{},[2489,2502,2503],{},"Treating every change as fatal."," Classify findings so harmless additions do not block reports.",[2486,2506,2507,2510],{},[2489,2508,2509],{},"Accepting rename suggestions automatically."," Confirm them; similar names can be different fields.",[2486,2512,2513,2516],{},[2489,2514,2515],{},"Keeping no history."," Save each month's snapshot to compare and to explain.",[25,2518,2520],{"id":2519},"conclusion","Conclusion",[10,2522,2523],{},"Schema drift — renamed, added or removed columns, type changes, new category values, rising blanks — is the quiet way recurring Excel reports go wrong. Describe each file in a small snapshot of columns, types, blank rates, categories and ranges; compare it with the previous month's to find structural changes, suggest renames by name similarity, and list per-column drift in plain words; then classify the findings and stop the pipeline when anything blocking appears. Save the snapshots, and the check doubles as a history of how the export has changed.",[25,2525,2527],{"id":2526},"frequently-asked-questions","Frequently asked questions",[10,2529,2530,2533],{},[2489,2531,2532],{},"What is schema drift in Excel files?","\nChanges in the structure or content of a recurring file, such as renamed or removed columns, new columns, type changes, new category values or more blanks, that can break or silently distort reports built on it.",[10,2535,2536,2539],{},[2489,2537,2538],{},"How do I compare the columns of two Excel files in Python?","\nRead both headers with pandas, take the set differences to find added and removed columns, and compare positions for columns present in both.",[10,2541,2542,2545],{},[2489,2543,2544],{},"How can I detect a renamed column?","\nWhen one column disappears and another appears in the same month, compare their names with difflib.SequenceMatcher and suggest pairs with a high similarity for a person to confirm.",[10,2547,2548,2551],{},[2489,2549,2550],{},"Should the report fail when the file changes?","\nOnly for blocking changes, such as missing required columns or non-numeric values in numeric columns. Treat other changes as warnings or information.",[25,2553,2555],{"id":2554},"related","Related",[2483,2557,2558,2564,2571,2578],{},[2486,2559,2560,2561,2563],{},"Up one level: ",[19,2562,22],{"href":21}," — checking data before it is used.",[2486,2565,2566,2570],{},[19,2567,2569],{"href":2568},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fcheck-excel-data-types-with-pandas\u002F","Check Excel Data Types with pandas"," — types within one file.",[2486,2572,2573,2577],{},[19,2574,2576],{"href":2575},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas\u002F","Profile an Excel Dataset with pandas"," — a fuller one-off description.",[2486,2579,2580,2584],{},[19,2581,2583],{"href":2582},"\u002Fadvanced-data-transformation-and-cleaning\u002Freading-excel-with-polars-and-arrow\u002Ffix-schema-and-type-errors-when-reading-excel-with-polars\u002F","Fix Schema and Type Errors When Reading Excel with Polars"," — drift meeting a strict reader.",[2586,2587,2588],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":35,"searchDepth":91,"depth":91,"links":2590},[2591,2592,2593,2594,2595,2596,2597,2598,2599,2600],{"id":27,"depth":91,"text":28},{"id":67,"depth":91,"text":68},{"id":604,"depth":91,"text":605},{"id":1043,"depth":91,"text":1044},{"id":1805,"depth":91,"text":1806},{"id":2469,"depth":91,"text":2470},{"id":2480,"depth":91,"text":2481},{"id":2519,"depth":91,"text":2520},{"id":2526,"depth":91,"text":2527},{"id":2554,"depth":91,"text":2555},"2026-10-02","Catch the silent changes in a recurring Excel export before they break a report: take a schema snapshot of each file — columns, order, types, blanks, categories and ranges — compare it with last month's, flag added, removed and probably renamed columns, type changes, new category values and shifted ranges, and stop the pipeline with a readable drift report when a change matters.","md",[2605,2607,2609,2611],{"q":2532,"a":2606},"Changes in the structure or content of a recurring file, such as renamed or removed columns, new columns, type changes, new category values or more blanks, that can break or silently distort reports built on it.",{"q":2538,"a":2608},"Read both headers with pandas, take the set differences to find added and removed columns, and compare positions for columns present in both.",{"q":2544,"a":2610},"When one column disappears and another appears in the same month, compare their names with difflib.SequenceMatcher and suggest pairs with a high similarity for a person to confirm.",{"q":2550,"a":2612},"Only for blocking changes, such as missing required columns or non-numeric values in numeric columns. Treat other changes as warnings or information.","detect schema drift between monthly excel files",null,{"breadcrumb":2616},[2617,2620,2623],{"name":2618,"item":2619},"Home","\u002F",{"name":2621,"item":2622},"Advanced Data Transformation and Cleaning for Excel Automation","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":22,"item":21},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files",{"title":2626,"description":2627},"Detect Schema Drift Between Monthly Excel Files in Python","Compare monthly Excel exports with pandas: schema snapshots saved as JSON, added, removed and renamed columns, dtype changes, new category values, range shifts and blank-rate jumps, with a drift report and a fail rule.","detect-schema-drift-between-monthly-excel-files","advanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-schema-drift-between-monthly-excel-files\u002Findex",[2631],"data-validation","how-to","WjlE384AGReZOmbjmfzWb8IdYpzoh0JCsjgciuuIO4o",[2635,2639],{"title":2636,"path":2637,"stem":2638,"children":-1},"Detect Possible Duplicate Invoices","\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-possible-duplicate-invoices","advanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fdetect-possible-duplicate-invoices\u002Findex",{"title":2640,"path":2641,"stem":2642,"children":-1},"Find Duplicate Rows in Excel with Python","\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Ffind-duplicate-rows-in-excel-with-python","advanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Ffind-duplicate-rows-in-excel-with-python\u002Findex",[2644,2646],{"path":2645,"title":2621},"\u002Fadvanced-data-transformation-and-cleaning",{"path":2647,"title":22},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python",[],[2650,2653,2656,2659,2662,2665],{"path":2651,"title":2652,"kind":2632},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Fdetect-outliers-in-excel-data-with-pandas","Detect Outliers in Excel Data with pandas",{"path":2654,"title":2655,"kind":2632},"\u002Fadvanced-data-transformation-and-cleaning\u002Fapplying-conditional-formatting-with-openpyxl\u002Fhighlight-cells-containing-text-with-openpyxl","Highlight Cells Containing Specific Text with openpyxl",{"path":2657,"title":2658,"kind":2632},"\u002Fadvanced-data-transformation-and-cleaning\u002Fcleaning-excel-data-with-pandas\u002Ffind-near-duplicate-names-within-one-excel-column","Find Near-Duplicate Names Within One Excel Column",{"path":2660,"title":2661,"kind":2632},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fread-dropdown-lists-from-an-existing-workbook","Read Dropdown Lists and Validation Rules from an Existing Workbook",{"path":2663,"title":2664,"kind":2632},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file","Find and Report Missing Values in an Excel File",{"path":2666,"title":2667,"kind":2632},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Fflag-rows-with-too-many-missing-values","Flag Rows with Too Many Missing Values",1790972098628]