[{"data":1,"prerenderedAt":2166},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fvlookup-equivalent-in-pandas-for-excel-files":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fvlookup-equivalent-in-pandas-for-excel-files":2159},{"id":4,"title":5,"body":6,"dateModified":2127,"datePublished":2127,"description":2128,"extension":2129,"faq":2130,"meta":2140,"navigation":218,"path":2151,"seo":2152,"slug":2155,"stem":2156,"type":2157,"__hash__":2158},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fvlookup-equivalent-in-pandas-for-excel-files\u002Findex.md","The VLOOKUP Equivalent in pandas for Excel Files",{"type":7,"value":8,"toc":2113},"minimark",[9,13,22,150,155,183,186,190,508,518,522,528,718,736,747,751,757,910,934,942,1015,1019,1026,1109,1121,1140,1230,1234,1240,1378,1386,1390,1400,1658,1785,1803,1807,1947,1951,1960,1972,1976,1997,2001,2021,2040,2049,2058,2062,2065,2074,2077,2109],[10,11,12],"p",{},"VLOOKUP is the function that made spreadsheets a database, and translating it is usually the first thing anyone does when moving a report to pandas. The translation is not one function but three, because VLOOKUP quietly does three different jobs: exact-match lookup, single-value translation, and — with its fourth argument set to TRUE — a banded approximate match that most people have used without noticing.",[10,14,15,16,21],{},"This guide covers all three, plus the two things pandas makes visible that Excel hides: which rows failed to match, and what happens when the lookup table has duplicate keys. It is part of ",[17,18,20],"a",{"href":19},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002F","Merging and Joining Excel DataFrames",".",[23,24,32,33,32,37,32,41,32,48,32,55,32,65,32,71,32,76,32,83,32,88,32,93,32,97,32,102,32,106,32,109,32,113,32,117,32,120,32,123,32,128,32,133,32,136,32,140,32,144,32,147],"svg",{"viewBox":25,"role":26,"ariaLabelledBy":27,"xmlns":30,"style":31},"0 0 740 250","img",[28,29],"vl-map-t","vl-map-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:740px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[34,35,36],"title",{"id":28},"Three jobs VLOOKUP does, and the pandas call for each",[38,39,40],"desc",{"id":29},"An exact-match lookup that returns several columns becomes merge with how left. A single key-to-value translation becomes map over a dictionary or Series. An approximate match, VLOOKUP's fourth argument set to TRUE, becomes merge_asof, which joins on the nearest key at or below the value.",[42,43],"rect",{"x":44,"y":44,"width":45,"height":46,"fill":47},"0","740","250","#ffffff",[49,50,54],"text",{"x":51,"y":52,"style":53},"370","30","font-size:12.5px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","One spreadsheet function, three different operations",[42,56],{"x":57,"y":58,"width":59,"height":60,"rx":61,"fill":62,"stroke":63,"style":64},"16","52","228","164","14","#ebebfd","var(--brand,#5b5cf0)","stroke-width:2px",[49,66,70],{"x":67,"y":68,"style":69},"130","80","font-size:12.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","exact match, many columns",[49,72,75],{"x":67,"y":73,"style":74},"110","font-size:11.5px;fill:var(--muted,#5b6780);text-anchor:middle","VLOOKUP(A2, tbl, 3, FALSE)",[77,78],"line",{"x1":79,"y1":80,"x2":81,"y2":80,"stroke":82},"44","124","216","var(--line,#cdd5e6)",[49,84,87],{"x":67,"y":85,"style":86},"150","font-size:12px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","df.merge(lookup, how=\"left\")",[49,89,92],{"x":67,"y":90,"style":91},"180","font-size:11px;fill:var(--text,#172033);text-anchor:middle","brings across every column",[49,94,96],{"x":67,"y":95,"style":91},"198","you name, in one pass",[42,98],{"x":99,"y":58,"width":59,"height":60,"rx":61,"fill":100,"stroke":101,"style":64},"256","#d9f4f1","var(--teal,#0f9488)",[49,103,105],{"x":51,"y":68,"style":104},"font-size:12.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","one value from a key",[49,107,108],{"x":51,"y":73,"style":74},"VLOOKUP(A2, tbl, 2, FALSE)",[77,110],{"x1":111,"y1":80,"x2":112,"y2":80,"stroke":82},"284","456",[49,114,116],{"x":51,"y":85,"style":115},"font-size:12px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","df[\"k\"].map(mapping)",[49,118,119],{"x":51,"y":90,"style":91},"faster, and cannot",[49,121,122],{"x":51,"y":95,"style":91},"multiply the rows",[42,124],{"x":125,"y":58,"width":59,"height":60,"rx":61,"fill":126,"stroke":127,"style":64},"496","#fdefd8","var(--gold,#b4740a)",[49,129,132],{"x":130,"y":68,"style":131},"610","font-size:12.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","nearest match below",[49,134,135],{"x":130,"y":73,"style":74},"VLOOKUP(A2, tbl, 2, TRUE)",[77,137],{"x1":138,"y1":80,"x2":139,"y2":80,"stroke":82},"524","696",[49,141,143],{"x":130,"y":85,"style":142},"font-size:12px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","pd.merge_asof(...)",[49,145,146],{"x":130,"y":90,"style":91},"rate bands, tiers,",[49,148,149],{"x":130,"y":95,"style":91},"most recent reading",[151,152,154],"h2",{"id":153},"prerequisites","Prerequisites",[156,157,162],"pre",{"className":158,"code":159,"language":160,"meta":161,"style":161},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash","",[163,164,165],"code",{"__ignoreMap":161},[166,167,169,173,177,180],"span",{"class":77,"line":168},1,[166,170,172],{"class":171},"sMTad","pip",[166,174,176],{"class":175},"srMev"," install",[166,178,179],{"class":175}," pandas",[166,181,182],{"class":175}," openpyxl\n",[10,184,185],{},"Two workbooks: the transactions you are enriching and the lookup table you are enriching them from. The examples create both.",[151,187,189],{"id":188},"step-1-build-the-two-tables","Step 1: Build the two tables",[156,191,195],{"className":192,"code":193,"language":194,"meta":161,"style":161},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\norders = pd.DataFrame({\n    \"order_id\": [1, 2, 3, 4, 5],\n    \"product_code\": [\"A-100\", \"B-220\", \"A-100\", \"Z-999\", \"C-310\"],\n    \"quantity\": [2, 1, 5, 3, 4],\n})\n\nproducts = pd.DataFrame({\n    \"product_code\": [\"A-100\", \"B-220\", \"C-310\", \"D-400\"],\n    \"product_name\": [\"Widget\", \"Gadget\", \"Sprocket\", \"Flange\"],\n    \"unit_price\": [19.99, 45.00, 7.25, 12.10],\n    \"category\": [\"Core\", \"Core\", \"Accessory\", \"Accessory\"],\n})\n\norders.to_excel(\"orders.xlsx\", index=False)\nproducts.to_excel(\"products.xlsx\", index=False)\n","python",[163,196,197,213,220,232,269,301,329,335,340,350,374,402,430,456,461,466,489],{"__ignoreMap":161},[166,198,199,203,207,210],{"class":77,"line":168},[166,200,202],{"class":201},"s-kum","import",[166,204,206],{"class":205},"skGVy"," pandas ",[166,208,209],{"class":201},"as",[166,211,212],{"class":205}," pd\n",[166,214,216],{"class":77,"line":215},2,[166,217,219],{"emptyLinePlaceholder":218},true,"\n",[166,221,223,226,229],{"class":77,"line":222},3,[166,224,225],{"class":205},"orders ",[166,227,228],{"class":201},"=",[166,230,231],{"class":205}," pd.DataFrame({\n",[166,233,235,238,241,245,248,251,253,256,258,261,263,266],{"class":77,"line":234},4,[166,236,237],{"class":175},"    \"order_id\"",[166,239,240],{"class":205},": [",[166,242,244],{"class":243},"sP0c6","1",[166,246,247],{"class":205},", ",[166,249,250],{"class":243},"2",[166,252,247],{"class":205},[166,254,255],{"class":243},"3",[166,257,247],{"class":205},[166,259,260],{"class":243},"4",[166,262,247],{"class":205},[166,264,265],{"class":243},"5",[166,267,268],{"class":205},"],\n",[166,270,272,275,277,280,282,285,287,289,291,294,296,299],{"class":77,"line":271},5,[166,273,274],{"class":175},"    \"product_code\"",[166,276,240],{"class":205},[166,278,279],{"class":175},"\"A-100\"",[166,281,247],{"class":205},[166,283,284],{"class":175},"\"B-220\"",[166,286,247],{"class":205},[166,288,279],{"class":175},[166,290,247],{"class":205},[166,292,293],{"class":175},"\"Z-999\"",[166,295,247],{"class":205},[166,297,298],{"class":175},"\"C-310\"",[166,300,268],{"class":205},[166,302,304,307,309,311,313,315,317,319,321,323,325,327],{"class":77,"line":303},6,[166,305,306],{"class":175},"    \"quantity\"",[166,308,240],{"class":205},[166,310,250],{"class":243},[166,312,247],{"class":205},[166,314,244],{"class":243},[166,316,247],{"class":205},[166,318,265],{"class":243},[166,320,247],{"class":205},[166,322,255],{"class":243},[166,324,247],{"class":205},[166,326,260],{"class":243},[166,328,268],{"class":205},[166,330,332],{"class":77,"line":331},7,[166,333,334],{"class":205},"})\n",[166,336,338],{"class":77,"line":337},8,[166,339,219],{"emptyLinePlaceholder":218},[166,341,343,346,348],{"class":77,"line":342},9,[166,344,345],{"class":205},"products ",[166,347,228],{"class":201},[166,349,231],{"class":205},[166,351,353,355,357,359,361,363,365,367,369,372],{"class":77,"line":352},10,[166,354,274],{"class":175},[166,356,240],{"class":205},[166,358,279],{"class":175},[166,360,247],{"class":205},[166,362,284],{"class":175},[166,364,247],{"class":205},[166,366,298],{"class":175},[166,368,247],{"class":205},[166,370,371],{"class":175},"\"D-400\"",[166,373,268],{"class":205},[166,375,377,380,382,385,387,390,392,395,397,400],{"class":77,"line":376},11,[166,378,379],{"class":175},"    \"product_name\"",[166,381,240],{"class":205},[166,383,384],{"class":175},"\"Widget\"",[166,386,247],{"class":205},[166,388,389],{"class":175},"\"Gadget\"",[166,391,247],{"class":205},[166,393,394],{"class":175},"\"Sprocket\"",[166,396,247],{"class":205},[166,398,399],{"class":175},"\"Flange\"",[166,401,268],{"class":205},[166,403,405,408,410,413,415,418,420,423,425,428],{"class":77,"line":404},12,[166,406,407],{"class":175},"    \"unit_price\"",[166,409,240],{"class":205},[166,411,412],{"class":243},"19.99",[166,414,247],{"class":205},[166,416,417],{"class":243},"45.00",[166,419,247],{"class":205},[166,421,422],{"class":243},"7.25",[166,424,247],{"class":205},[166,426,427],{"class":243},"12.10",[166,429,268],{"class":205},[166,431,433,436,438,441,443,445,447,450,452,454],{"class":77,"line":432},13,[166,434,435],{"class":175},"    \"category\"",[166,437,240],{"class":205},[166,439,440],{"class":175},"\"Core\"",[166,442,247],{"class":205},[166,444,440],{"class":175},[166,446,247],{"class":205},[166,448,449],{"class":175},"\"Accessory\"",[166,451,247],{"class":205},[166,453,449],{"class":175},[166,455,268],{"class":205},[166,457,459],{"class":77,"line":458},14,[166,460,334],{"class":205},[166,462,464],{"class":77,"line":463},15,[166,465,219],{"emptyLinePlaceholder":218},[166,467,469,472,475,477,481,483,486],{"class":77,"line":468},16,[166,470,471],{"class":205},"orders.to_excel(",[166,473,474],{"class":175},"\"orders.xlsx\"",[166,476,247],{"class":205},[166,478,480],{"class":479},"sa561","index",[166,482,228],{"class":201},[166,484,485],{"class":243},"False",[166,487,488],{"class":205},")\n",[166,490,492,495,498,500,502,504,506],{"class":77,"line":491},17,[166,493,494],{"class":205},"products.to_excel(",[166,496,497],{"class":175},"\"products.xlsx\"",[166,499,247],{"class":205},[166,501,480],{"class":479},[166,503,228],{"class":201},[166,505,485],{"class":243},[166,507,488],{"class":205},[10,509,510,513,514,517],{},[163,511,512],{},"Z-999"," is deliberately absent from the lookup table — it is the row that would produce ",[163,515,516],{},"#N\u002FA"," in a spreadsheet, and the one this guide cares about most.",[151,519,521],{"id":520},"step-2-the-direct-translation-merge","Step 2: The direct translation — merge",[10,523,524,527],{},[163,525,526],{},"=VLOOKUP(B2, products!A:D, 2, FALSE)"," becomes a left merge, and unlike VLOOKUP it can bring several columns at once:",[156,529,531],{"className":192,"code":530,"language":194,"meta":161,"style":161},"orders = pd.read_excel(\"orders.xlsx\", dtype={\"product_code\": str})\nproducts = pd.read_excel(\"products.xlsx\", dtype={\"product_code\": str})\n\nenriched = orders.merge(\n    products[[\"product_code\", \"product_name\", \"unit_price\", \"category\"]],\n    on=\"product_code\",\n    how=\"left\",              # keep every order, matched or not\n    validate=\"m:1\",          # many orders per product, one row per product\n)\nenriched[\"line_total\"] = enriched[\"quantity\"] * enriched[\"unit_price\"]\nprint(enriched)\n",[163,532,533,565,591,595,605,630,642,659,675,679,710],{"__ignoreMap":161},[166,534,535,537,539,542,544,546,549,551,554,557,560,563],{"class":77,"line":168},[166,536,225],{"class":205},[166,538,228],{"class":201},[166,540,541],{"class":205}," pd.read_excel(",[166,543,474],{"class":175},[166,545,247],{"class":205},[166,547,548],{"class":479},"dtype",[166,550,228],{"class":201},[166,552,553],{"class":205},"{",[166,555,556],{"class":175},"\"product_code\"",[166,558,559],{"class":205},": ",[166,561,562],{"class":243},"str",[166,564,334],{"class":205},[166,566,567,569,571,573,575,577,579,581,583,585,587,589],{"class":77,"line":215},[166,568,345],{"class":205},[166,570,228],{"class":201},[166,572,541],{"class":205},[166,574,497],{"class":175},[166,576,247],{"class":205},[166,578,548],{"class":479},[166,580,228],{"class":201},[166,582,553],{"class":205},[166,584,556],{"class":175},[166,586,559],{"class":205},[166,588,562],{"class":243},[166,590,334],{"class":205},[166,592,593],{"class":77,"line":222},[166,594,219],{"emptyLinePlaceholder":218},[166,596,597,600,602],{"class":77,"line":234},[166,598,599],{"class":205},"enriched ",[166,601,228],{"class":201},[166,603,604],{"class":205}," orders.merge(\n",[166,606,607,610,612,614,617,619,622,624,627],{"class":77,"line":271},[166,608,609],{"class":205},"    products[[",[166,611,556],{"class":175},[166,613,247],{"class":205},[166,615,616],{"class":175},"\"product_name\"",[166,618,247],{"class":205},[166,620,621],{"class":175},"\"unit_price\"",[166,623,247],{"class":205},[166,625,626],{"class":175},"\"category\"",[166,628,629],{"class":205},"]],\n",[166,631,632,635,637,639],{"class":77,"line":303},[166,633,634],{"class":479},"    on",[166,636,228],{"class":201},[166,638,556],{"class":175},[166,640,641],{"class":205},",\n",[166,643,644,647,649,652,655],{"class":77,"line":331},[166,645,646],{"class":479},"    how",[166,648,228],{"class":201},[166,650,651],{"class":175},"\"left\"",[166,653,654],{"class":205},",              ",[166,656,658],{"class":657},"s-wDw","# keep every order, matched or not\n",[166,660,661,664,666,669,672],{"class":77,"line":337},[166,662,663],{"class":479},"    validate",[166,665,228],{"class":201},[166,667,668],{"class":175},"\"m:1\"",[166,670,671],{"class":205},",          ",[166,673,674],{"class":657},"# many orders per product, one row per product\n",[166,676,677],{"class":77,"line":342},[166,678,488],{"class":205},[166,680,681,684,687,690,692,695,698,700,703,705,707],{"class":77,"line":352},[166,682,683],{"class":205},"enriched[",[166,685,686],{"class":175},"\"line_total\"",[166,688,689],{"class":205},"] ",[166,691,228],{"class":201},[166,693,694],{"class":205}," enriched[",[166,696,697],{"class":175},"\"quantity\"",[166,699,689],{"class":205},[166,701,702],{"class":201},"*",[166,704,694],{"class":205},[166,706,621],{"class":175},[166,708,709],{"class":205},"]\n",[166,711,712,715],{"class":77,"line":376},[166,713,714],{"class":243},"print",[166,716,717],{"class":205},"(enriched)\n",[10,719,720,721,724,725,727,728,731,732,735],{},"Three arguments carry the meaning. ",[163,722,723],{},"how=\"left\""," keeps every order whether or not it matched — the equivalent of VLOOKUP leaving ",[163,726,516],{}," rather than dropping the row. ",[163,729,730],{},"on=\"product_code\""," is the lookup key. And ",[163,733,734],{},"validate=\"m:1\""," is the guard with no spreadsheet equivalent: it asserts that the lookup table has one row per key, and raises immediately if it does not.",[10,737,738,739,742,743,746],{},"Reading both key columns with ",[163,740,741],{},"dtype=str"," prevents the classic silent failure — one file storing ",[163,744,745],{},"00123"," as text and the other as a number, so nothing matches and every row comes back empty.",[151,748,750],{"id":749},"step-3-find-the-rows-that-did-not-match","Step 3: Find the rows that did not match",[10,752,753,754,756],{},"In Excel you scan for ",[163,755,516],{},". In pandas you ask directly:",[156,758,760],{"className":192,"code":759,"language":194,"meta":161,"style":161},"checked = orders.merge(products, on=\"product_code\", how=\"left\", indicator=True)\n\nmissing = checked[checked[\"_merge\"] == \"left_only\"]\nif not missing.empty:\n    print(f\"{len(missing)} order(s) with no product record:\")\n    print(missing[[\"order_id\", \"product_code\"]].to_string(index=False))\n# 1 order(s) with no product record:\n#  order_id product_code\n#         4        Z-999\n",[163,761,762,800,804,827,838,869,895,900,905],{"__ignoreMap":161},[166,763,764,767,769,772,775,777,779,781,784,786,788,790,793,795,798],{"class":77,"line":168},[166,765,766],{"class":205},"checked ",[166,768,228],{"class":201},[166,770,771],{"class":205}," orders.merge(products, ",[166,773,774],{"class":479},"on",[166,776,228],{"class":201},[166,778,556],{"class":175},[166,780,247],{"class":205},[166,782,783],{"class":479},"how",[166,785,228],{"class":201},[166,787,651],{"class":175},[166,789,247],{"class":205},[166,791,792],{"class":479},"indicator",[166,794,228],{"class":201},[166,796,797],{"class":243},"True",[166,799,488],{"class":205},[166,801,802],{"class":77,"line":215},[166,803,219],{"emptyLinePlaceholder":218},[166,805,806,809,811,814,817,819,822,825],{"class":77,"line":222},[166,807,808],{"class":205},"missing ",[166,810,228],{"class":201},[166,812,813],{"class":205}," checked[checked[",[166,815,816],{"class":175},"\"_merge\"",[166,818,689],{"class":205},[166,820,821],{"class":201},"==",[166,823,824],{"class":175}," \"left_only\"",[166,826,709],{"class":205},[166,828,829,832,835],{"class":77,"line":234},[166,830,831],{"class":201},"if",[166,833,834],{"class":201}," not",[166,836,837],{"class":205}," missing.empty:\n",[166,839,840,843,846,849,852,855,858,861,864,867],{"class":77,"line":271},[166,841,842],{"class":243},"    print",[166,844,845],{"class":205},"(",[166,847,848],{"class":201},"f",[166,850,851],{"class":175},"\"",[166,853,553],{"class":854},"sSjpA",[166,856,857],{"class":243},"len",[166,859,860],{"class":205},"(missing)",[166,862,863],{"class":854},"}",[166,865,866],{"class":175}," order(s) with no product record:\"",[166,868,488],{"class":205},[166,870,871,873,876,879,881,883,886,888,890,892],{"class":77,"line":303},[166,872,842],{"class":243},[166,874,875],{"class":205},"(missing[[",[166,877,878],{"class":175},"\"order_id\"",[166,880,247],{"class":205},[166,882,556],{"class":175},[166,884,885],{"class":205},"]].to_string(",[166,887,480],{"class":479},[166,889,228],{"class":201},[166,891,485],{"class":243},[166,893,894],{"class":205},"))\n",[166,896,897],{"class":77,"line":331},[166,898,899],{"class":657},"# 1 order(s) with no product record:\n",[166,901,902],{"class":77,"line":337},[166,903,904],{"class":657},"#  order_id product_code\n",[166,906,907],{"class":77,"line":342},[166,908,909],{"class":657},"#         4        Z-999\n",[10,911,912,915,916,919,920,247,923,926,927,930,931,933],{},[163,913,914],{},"indicator=True"," adds a ",[163,917,918],{},"_merge"," column whose values are ",[163,921,922],{},"both",[163,924,925],{},"left_only"," or ",[163,928,929],{},"right_only",". That single column turns \"some rows have ",[163,932,516],{}," somewhere\" into a list of exactly which orders reference an unknown product — which is the report someone can act on.",[10,935,936,937,941],{},"In a scheduled job, decide deliberately what a miss means. A handful of unmatched rows in a million might be acceptable and worth logging; a fifth of the file failing to match usually means the two exports are from different periods, and the job should stop rather than publish a report with a fifth of its revenue missing. That check belongs with the rest of the ",[17,938,940],{"href":939},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002F","validation"," work:",[156,943,945],{"className":192,"code":944,"language":194,"meta":161,"style":161},"match_rate = (checked[\"_merge\"] == \"both\").mean()\nif match_rate \u003C 0.95:\n    raise ValueError(f\"only {match_rate:.1%} of orders matched a product\")\n",[163,946,947,969,985],{"__ignoreMap":161},[166,948,949,952,954,957,959,961,963,966],{"class":77,"line":168},[166,950,951],{"class":205},"match_rate ",[166,953,228],{"class":201},[166,955,956],{"class":205}," (checked[",[166,958,816],{"class":175},[166,960,689],{"class":205},[166,962,821],{"class":201},[166,964,965],{"class":175}," \"both\"",[166,967,968],{"class":205},").mean()\n",[166,970,971,973,976,979,982],{"class":77,"line":215},[166,972,831],{"class":201},[166,974,975],{"class":205}," match_rate ",[166,977,978],{"class":201},"\u003C",[166,980,981],{"class":243}," 0.95",[166,983,984],{"class":205},":\n",[166,986,987,990,993,995,997,1000,1002,1005,1008,1010,1013],{"class":77,"line":222},[166,988,989],{"class":201},"    raise",[166,991,992],{"class":243}," ValueError",[166,994,845],{"class":205},[166,996,848],{"class":201},[166,998,999],{"class":175},"\"only ",[166,1001,553],{"class":854},[166,1003,1004],{"class":205},"match_rate",[166,1006,1007],{"class":201},":.1%",[166,1009,863],{"class":854},[166,1011,1012],{"class":175}," of orders matched a product\"",[166,1014,488],{"class":205},[151,1016,1018],{"id":1017},"step-4-use-map-for-a-single-value-lookup","Step 4: Use map for a single-value lookup",[10,1020,1021,1022,1025],{},"When you only need one field, ",[163,1023,1024],{},"map"," is shorter, faster and structurally incapable of duplicating rows:",[156,1027,1029],{"className":192,"code":1028,"language":194,"meta":161,"style":161},"price_by_code = products.set_index(\"product_code\")[\"unit_price\"]\n\norders[\"unit_price\"] = orders[\"product_code\"].map(price_by_code)\norders[\"category\"] = orders[\"product_code\"].map(\n    products.set_index(\"product_code\")[\"category\"]).fillna(\"Unknown\")\n",[163,1030,1031,1050,1054,1073,1090],{"__ignoreMap":161},[166,1032,1033,1036,1038,1041,1043,1046,1048],{"class":77,"line":168},[166,1034,1035],{"class":205},"price_by_code ",[166,1037,228],{"class":201},[166,1039,1040],{"class":205}," products.set_index(",[166,1042,556],{"class":175},[166,1044,1045],{"class":205},")[",[166,1047,621],{"class":175},[166,1049,709],{"class":205},[166,1051,1052],{"class":77,"line":215},[166,1053,219],{"emptyLinePlaceholder":218},[166,1055,1056,1059,1061,1063,1065,1068,1070],{"class":77,"line":222},[166,1057,1058],{"class":205},"orders[",[166,1060,621],{"class":175},[166,1062,689],{"class":205},[166,1064,228],{"class":201},[166,1066,1067],{"class":205}," orders[",[166,1069,556],{"class":175},[166,1071,1072],{"class":205},"].map(price_by_code)\n",[166,1074,1075,1077,1079,1081,1083,1085,1087],{"class":77,"line":234},[166,1076,1058],{"class":205},[166,1078,626],{"class":175},[166,1080,689],{"class":205},[166,1082,228],{"class":201},[166,1084,1067],{"class":205},[166,1086,556],{"class":175},[166,1088,1089],{"class":205},"].map(\n",[166,1091,1092,1095,1097,1099,1101,1104,1107],{"class":77,"line":271},[166,1093,1094],{"class":205},"    products.set_index(",[166,1096,556],{"class":175},[166,1098,1045],{"class":205},[166,1100,626],{"class":175},[166,1102,1103],{"class":205},"]).fillna(",[166,1105,1106],{"class":175},"\"Unknown\"",[166,1108,488],{"class":205},[10,1110,1111,1113,1114,1117,1118,1120],{},[163,1112,1024],{}," accepts a dictionary or a Series indexed by the key, returns ",[163,1115,1116],{},"NaN"," for anything unmatched, and — crucially — always returns exactly as many values as it received. A merge against a lookup table with an accidental duplicate key silently produces extra rows; ",[163,1119,1024],{}," cannot, which makes it the safer choice when you genuinely need one column.",[10,1122,1123,1126,1127,1130,1131,1135,1136,1139],{},[163,1124,1125],{},"fillna(\"Unknown\")"," is the equivalent of wrapping the whole thing in ",[163,1128,1129],{},"IFERROR",", and it is worth being deliberate about: filling a missing ",[1132,1133,1134],"em",{},"price"," with zero would quietly understate a total, while filling a missing ",[1132,1137,1138],{},"category label"," is harmless.",[23,1141,32,1146,32,1149,32,1152,32,1155,32,1160,32,1165,32,1170,32,1174,32,1179,32,1182,32,1186,32,1190,32,1195,32,1200,32,1204,32,1206,32,1209,32,1212,32,1215,32,1218,32,1225],{"viewBox":1142,"role":26,"ariaLabelledBy":1143,"xmlns":30,"style":31},"0 0 740 246",[1144,1145],"vl-dupe-t","vl-dupe-d",[34,1147,1148],{"id":1144},"What a duplicate lookup key does to a merge",[38,1150,1151],{"id":1145},"VLOOKUP returns the first match and hides the duplicate entirely. A merge returns one output row per matching pair, so a lookup table with the same key twice doubles those orders and inflates every total downstream. Passing validate m to 1 makes pandas raise instead of silently multiplying the rows.",[42,1153],{"x":44,"y":44,"width":45,"height":1154,"fill":47},"246",[49,1156,1159],{"x":1157,"y":1158,"style":131},"184","32","VLOOKUP: first match wins",[42,1161],{"x":1162,"y":58,"width":1163,"height":52,"rx":1164,"fill":126,"stroke":127},"24","140","6",[49,1166,1169],{"x":1167,"y":1168,"style":91},"94","72","A-100 → 19.99",[42,1171],{"x":1162,"y":1172,"width":1163,"height":52,"rx":1164,"fill":1173,"stroke":82},"88","#f0f2f5",[49,1175,1178],{"x":1167,"y":1176,"style":1177},"108","font-size:11px;fill:var(--muted,#5b6780);text-anchor:middle","A-100 → 24.50",[49,1180,1181],{"x":1157,"y":1163,"style":1177},"the second row is never seen",[49,1183,1185],{"x":1157,"y":60,"style":1184},"font-size:11.5px;fill:var(--text,#172033);text-anchor:middle","row count unchanged, and the",[49,1187,1189],{"x":1157,"y":1188,"style":1184},"182","disagreement stays hidden",[49,1191,1194],{"x":1192,"y":1158,"style":1193},"556","font-size:12.5px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","merge: one row per pair",[42,1196],{"x":1197,"y":58,"width":1163,"height":52,"rx":1164,"fill":1198,"stroke":1199},"396","#fce9e9","var(--accent-ink,#be185d)",[49,1201,1203],{"x":1202,"y":1168,"style":91},"466","order 1 → 19.99",[42,1205],{"x":1197,"y":1172,"width":1163,"height":52,"rx":1164,"fill":1198,"stroke":1199},[49,1207,1208],{"x":1202,"y":1176,"style":91},"order 1 → 24.50",[49,1210,1211],{"x":1192,"y":1163,"style":1177},"one order became two rows",[49,1213,1214],{"x":1192,"y":60,"style":1184},"every total downstream is",[49,1216,1217],{"x":1192,"y":1188,"style":1184},"now overstated",[42,1219],{"x":1220,"y":1221,"width":1222,"height":1223,"rx":1224,"fill":100,"stroke":101,"style":64},"120","200","500","36","10",[49,1226,1229],{"x":51,"y":1227,"style":1228},"223","font-size:11.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","validate=\"m:1\" turns the silent inflation into an immediate error",[151,1231,1233],{"id":1232},"step-5-handle-duplicate-keys-deliberately","Step 5: Handle duplicate keys deliberately",[10,1235,1236,1237,1239],{},"If ",[163,1238,734],{}," raises, the lookup table has the same key more than once. Find them and decide what they mean before working around them:",[156,1241,1243],{"className":192,"code":1242,"language":194,"meta":161,"style":161},"dupes = products[products[\"product_code\"].duplicated(keep=False)]\nif not dupes.empty:\n    print(dupes.sort_values(\"product_code\").to_string(index=False))\n\n# Then choose ONE of these, deliberately:\nlatest = (products.sort_values(\"valid_from\")\n                  .drop_duplicates(\"product_code\", keep=\"last\"))     # newest wins\n# or aggregate, when several rows are all legitimate:\naveraged = products.groupby(\"product_code\", as_index=False)[\"unit_price\"].mean()\n",[163,1244,1245,1270,1279,1299,1303,1308,1323,1345,1350],{"__ignoreMap":161},[166,1246,1247,1250,1252,1255,1257,1260,1263,1265,1267],{"class":77,"line":168},[166,1248,1249],{"class":205},"dupes ",[166,1251,228],{"class":201},[166,1253,1254],{"class":205}," products[products[",[166,1256,556],{"class":175},[166,1258,1259],{"class":205},"].duplicated(",[166,1261,1262],{"class":479},"keep",[166,1264,228],{"class":201},[166,1266,485],{"class":243},[166,1268,1269],{"class":205},")]\n",[166,1271,1272,1274,1276],{"class":77,"line":215},[166,1273,831],{"class":201},[166,1275,834],{"class":201},[166,1277,1278],{"class":205}," dupes.empty:\n",[166,1280,1281,1283,1286,1288,1291,1293,1295,1297],{"class":77,"line":222},[166,1282,842],{"class":243},[166,1284,1285],{"class":205},"(dupes.sort_values(",[166,1287,556],{"class":175},[166,1289,1290],{"class":205},").to_string(",[166,1292,480],{"class":479},[166,1294,228],{"class":201},[166,1296,485],{"class":243},[166,1298,894],{"class":205},[166,1300,1301],{"class":77,"line":234},[166,1302,219],{"emptyLinePlaceholder":218},[166,1304,1305],{"class":77,"line":271},[166,1306,1307],{"class":657},"# Then choose ONE of these, deliberately:\n",[166,1309,1310,1313,1315,1318,1321],{"class":77,"line":303},[166,1311,1312],{"class":205},"latest ",[166,1314,228],{"class":201},[166,1316,1317],{"class":205}," (products.sort_values(",[166,1319,1320],{"class":175},"\"valid_from\"",[166,1322,488],{"class":205},[166,1324,1325,1328,1330,1332,1334,1336,1339,1342],{"class":77,"line":331},[166,1326,1327],{"class":205},"                  .drop_duplicates(",[166,1329,556],{"class":175},[166,1331,247],{"class":205},[166,1333,1262],{"class":479},[166,1335,228],{"class":201},[166,1337,1338],{"class":175},"\"last\"",[166,1340,1341],{"class":205},"))     ",[166,1343,1344],{"class":657},"# newest wins\n",[166,1346,1347],{"class":77,"line":337},[166,1348,1349],{"class":657},"# or aggregate, when several rows are all legitimate:\n",[166,1351,1352,1355,1357,1360,1362,1364,1367,1369,1371,1373,1375],{"class":77,"line":342},[166,1353,1354],{"class":205},"averaged ",[166,1356,228],{"class":201},[166,1358,1359],{"class":205}," products.groupby(",[166,1361,556],{"class":175},[166,1363,247],{"class":205},[166,1365,1366],{"class":479},"as_index",[166,1368,228],{"class":201},[166,1370,485],{"class":243},[166,1372,1045],{"class":205},[166,1374,621],{"class":175},[166,1376,1377],{"class":205},"].mean()\n",[10,1379,1380,1381,1385],{},"The reason to look first is that duplicates usually mean something: two price versions with different effective dates, a product listed under two categories, or an export that ran twice. Silently taking the first — which is what VLOOKUP does — hides whichever of those it is. ",[17,1382,1384],{"href":1383},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Ffind-duplicate-rows-in-excel-with-python\u002F","Find Duplicate Rows in Excel with Python"," covers reporting them back to whoever owns the data.",[151,1387,1389],{"id":1388},"step-6-the-approximate-match-merge_asof","Step 6: The approximate match — merge_asof",[10,1391,1392,1395,1396,1399],{},[163,1393,1394],{},"VLOOKUP(value, table, 2, TRUE)"," finds the largest key at or below the value, which is how rate bands, commission tiers and postage brackets are built. In pandas that is ",[163,1397,1398],{},"merge_asof",":",[156,1401,1403],{"className":192,"code":1402,"language":194,"meta":161,"style":161},"bands = pd.DataFrame({\n    \"threshold\": [0, 1_000, 5_000, 20_000],\n    \"rate\": [0.00, 0.02, 0.035, 0.05],\n})\n\ndeals = pd.DataFrame({\"deal_id\": [1, 2, 3, 4],\n                      \"value\": [450, 3_200, 18_000, 92_000]})\n\nbanded = pd.merge_asof(\n    deals.sort_values(\"value\"),\n    bands.sort_values(\"threshold\"),\n    left_on=\"value\", right_on=\"threshold\",\n    direction=\"backward\",            # the largest threshold \u003C= value\n)\nprint(banded[[\"deal_id\", \"value\", \"rate\"]])\n#    deal_id  value   rate\n# 0        1    450  0.000\n# 1        2   3200  0.020\n# 2        3  18000  0.035\n# 3        4  92000  0.050\n",[163,1404,1405,1414,1440,1467,1471,1475,1506,1534,1538,1548,1559,1569,1589,1605,1609,1630,1635,1640,1646,1652],{"__ignoreMap":161},[166,1406,1407,1410,1412],{"class":77,"line":168},[166,1408,1409],{"class":205},"bands ",[166,1411,228],{"class":201},[166,1413,231],{"class":205},[166,1415,1416,1419,1421,1423,1425,1428,1430,1433,1435,1438],{"class":77,"line":215},[166,1417,1418],{"class":175},"    \"threshold\"",[166,1420,240],{"class":205},[166,1422,44],{"class":243},[166,1424,247],{"class":205},[166,1426,1427],{"class":243},"1_000",[166,1429,247],{"class":205},[166,1431,1432],{"class":243},"5_000",[166,1434,247],{"class":205},[166,1436,1437],{"class":243},"20_000",[166,1439,268],{"class":205},[166,1441,1442,1445,1447,1450,1452,1455,1457,1460,1462,1465],{"class":77,"line":222},[166,1443,1444],{"class":175},"    \"rate\"",[166,1446,240],{"class":205},[166,1448,1449],{"class":243},"0.00",[166,1451,247],{"class":205},[166,1453,1454],{"class":243},"0.02",[166,1456,247],{"class":205},[166,1458,1459],{"class":243},"0.035",[166,1461,247],{"class":205},[166,1463,1464],{"class":243},"0.05",[166,1466,268],{"class":205},[166,1468,1469],{"class":77,"line":234},[166,1470,334],{"class":205},[166,1472,1473],{"class":77,"line":271},[166,1474,219],{"emptyLinePlaceholder":218},[166,1476,1477,1480,1482,1485,1488,1490,1492,1494,1496,1498,1500,1502,1504],{"class":77,"line":303},[166,1478,1479],{"class":205},"deals ",[166,1481,228],{"class":201},[166,1483,1484],{"class":205}," pd.DataFrame({",[166,1486,1487],{"class":175},"\"deal_id\"",[166,1489,240],{"class":205},[166,1491,244],{"class":243},[166,1493,247],{"class":205},[166,1495,250],{"class":243},[166,1497,247],{"class":205},[166,1499,255],{"class":243},[166,1501,247],{"class":205},[166,1503,260],{"class":243},[166,1505,268],{"class":205},[166,1507,1508,1511,1513,1516,1518,1521,1523,1526,1528,1531],{"class":77,"line":331},[166,1509,1510],{"class":175},"                      \"value\"",[166,1512,240],{"class":205},[166,1514,1515],{"class":243},"450",[166,1517,247],{"class":205},[166,1519,1520],{"class":243},"3_200",[166,1522,247],{"class":205},[166,1524,1525],{"class":243},"18_000",[166,1527,247],{"class":205},[166,1529,1530],{"class":243},"92_000",[166,1532,1533],{"class":205},"]})\n",[166,1535,1536],{"class":77,"line":337},[166,1537,219],{"emptyLinePlaceholder":218},[166,1539,1540,1543,1545],{"class":77,"line":342},[166,1541,1542],{"class":205},"banded ",[166,1544,228],{"class":201},[166,1546,1547],{"class":205}," pd.merge_asof(\n",[166,1549,1550,1553,1556],{"class":77,"line":352},[166,1551,1552],{"class":205},"    deals.sort_values(",[166,1554,1555],{"class":175},"\"value\"",[166,1557,1558],{"class":205},"),\n",[166,1560,1561,1564,1567],{"class":77,"line":376},[166,1562,1563],{"class":205},"    bands.sort_values(",[166,1565,1566],{"class":175},"\"threshold\"",[166,1568,1558],{"class":205},[166,1570,1571,1574,1576,1578,1580,1583,1585,1587],{"class":77,"line":404},[166,1572,1573],{"class":479},"    left_on",[166,1575,228],{"class":201},[166,1577,1555],{"class":175},[166,1579,247],{"class":205},[166,1581,1582],{"class":479},"right_on",[166,1584,228],{"class":201},[166,1586,1566],{"class":175},[166,1588,641],{"class":205},[166,1590,1591,1594,1596,1599,1602],{"class":77,"line":432},[166,1592,1593],{"class":479},"    direction",[166,1595,228],{"class":201},[166,1597,1598],{"class":175},"\"backward\"",[166,1600,1601],{"class":205},",            ",[166,1603,1604],{"class":657},"# the largest threshold \u003C= value\n",[166,1606,1607],{"class":77,"line":458},[166,1608,488],{"class":205},[166,1610,1611,1613,1616,1618,1620,1622,1624,1627],{"class":77,"line":463},[166,1612,714],{"class":243},[166,1614,1615],{"class":205},"(banded[[",[166,1617,1487],{"class":175},[166,1619,247],{"class":205},[166,1621,1555],{"class":175},[166,1623,247],{"class":205},[166,1625,1626],{"class":175},"\"rate\"",[166,1628,1629],{"class":205},"]])\n",[166,1631,1632],{"class":77,"line":468},[166,1633,1634],{"class":657},"#    deal_id  value   rate\n",[166,1636,1637],{"class":77,"line":491},[166,1638,1639],{"class":657},"# 0        1    450  0.000\n",[166,1641,1643],{"class":77,"line":1642},18,[166,1644,1645],{"class":657},"# 1        2   3200  0.020\n",[166,1647,1649],{"class":77,"line":1648},19,[166,1650,1651],{"class":657},"# 2        3  18000  0.035\n",[166,1653,1655],{"class":77,"line":1654},20,[166,1656,1657],{"class":657},"# 3        4  92000  0.050\n",[23,1659,32,1664,32,1667,32,1670,32,1673,32,1676,32,1681,32,1686,32,1690,32,1693,32,1698,32,1701,32,1704,32,1709,32,1712,32,1716,32,1721,32,1724,32,1727,32,1731,32,1734,32,1738,32,1741,32,1745,32,1748,32,1752,32,1756,32,1759,32,1763,32,1767,32,1770,32,1776,32,1778,32,1780,32,1782],{"viewBox":1660,"role":26,"ariaLabelledBy":1661,"xmlns":30,"style":31},"0 0 740 220",[1662,1663],"vl-band-t","vl-band-d",[34,1665,1666],{"id":1662},"How a backward merge_asof assigns each deal to a band",[38,1668,1669],{"id":1663},"The thresholds zero, one thousand, five thousand and twenty thousand divide the number line into four bands with rates of nought, two, three and a half and five percent. Each deal is matched to the largest threshold at or below its value, so a deal of three thousand two hundred falls in the one-thousand band and takes the two percent rate.",[42,1671],{"x":44,"y":44,"width":45,"height":1672,"fill":47},"220",[49,1674,1675],{"x":51,"y":52,"style":53},"direction=\"backward\" — the largest threshold at or below the value",[42,1677],{"x":1162,"y":58,"width":1678,"height":1679,"rx":1680,"fill":1173,"stroke":82,"style":64},"166","46","8",[49,1682,1685],{"x":1683,"y":1168,"style":1684},"107","font-size:11px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","0 to 999",[49,1687,1689],{"x":1683,"y":1688,"style":1177},"90","rate 0.0%",[42,1691],{"x":1692,"y":58,"width":1678,"height":1679,"rx":1680,"fill":100,"stroke":101,"style":64},"196",[49,1694,1697],{"x":1695,"y":1168,"style":1696},"279","font-size:11px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","1,000 to 4,999",[49,1699,1700],{"x":1695,"y":1688,"style":91},"rate 2.0%",[42,1702],{"x":1703,"y":58,"width":1678,"height":1679,"rx":1680,"fill":62,"stroke":63,"style":64},"368",[49,1705,1708],{"x":1706,"y":1168,"style":1707},"451","font-size:11px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","5,000 to 19,999",[49,1710,1711],{"x":1706,"y":1688,"style":91},"rate 3.5%",[42,1713],{"x":1714,"y":58,"width":1715,"height":1679,"rx":1680,"fill":126,"stroke":127,"style":64},"540","176",[49,1717,1720],{"x":1718,"y":1168,"style":1719},"628","font-size:11px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","20,000 and above",[49,1722,1723],{"x":1718,"y":1688,"style":91},"rate 5.0%",[77,1725],{"x1":1162,"y1":1220,"x2":1726,"y2":1220,"stroke":82,"style":64},"716",[1728,1729],"circle",{"cx":1688,"cy":1220,"r":1164,"fill":1730},"#5b6780",[49,1732,1515],{"x":1688,"y":1733,"style":1177},"146",[49,1735,1737],{"x":1688,"y":60,"style":1736},"font-size:10.5px;fill:var(--muted,#5b6780);text-anchor:middle","deal 1",[1728,1739],{"cx":1695,"cy":1220,"r":1164,"fill":1740},"#0f766e",[49,1742,1744],{"x":1695,"y":1733,"style":1743},"font-size:11px;fill:var(--teal-ink,#0b6157);text-anchor:middle","3,200",[49,1746,1747],{"x":1695,"y":60,"style":1736},"deal 2",[1728,1749],{"cx":1750,"cy":1220,"r":1164,"fill":1751},"470","#5b5cf0",[49,1753,1755],{"x":1750,"y":1733,"style":1754},"font-size:11px;fill:var(--brand-strong,#4338ca);text-anchor:middle","18,000",[49,1757,1758],{"x":1750,"y":60,"style":1736},"deal 3",[1728,1760],{"cx":1761,"cy":1220,"r":1164,"fill":1762},"660","#8a5808",[49,1764,1766],{"x":1761,"y":1733,"style":1765},"font-size:11px;fill:var(--gold-ink,#7a4e06);text-anchor:middle","92,000",[49,1768,1769],{"x":1761,"y":60,"style":1736},"deal 4",[77,1771],{"x1":1688,"y1":1772,"x2":1688,"y2":1773,"stroke":1774,"style":1775},"114","100","var(--muted,#5b6780)","stroke-width:1.5px",[77,1777],{"x1":1695,"y1":1772,"x2":1695,"y2":1773,"stroke":101,"style":1775},[77,1779],{"x1":1750,"y1":1772,"x2":1750,"y2":1773,"stroke":63,"style":1775},[77,1781],{"x1":1761,"y1":1772,"x2":1761,"y2":1773,"stroke":127,"style":1775},[49,1783,1784],{"x":51,"y":1692,"style":1177},"A deal never falls between bands — every value maps to exactly one row of the rate table",[10,1786,1787,1788,1790,1791,1794,1795,1798,1799,1802],{},"Both frames must be sorted on the join key or ",[163,1789,1398],{}," raises — that requirement is the same one that makes VLOOKUP's approximate mode return nonsense on an unsorted table, except pandas tells you instead of guessing. ",[163,1792,1793],{},"direction=\"backward\""," is the VLOOKUP-TRUE behaviour; ",[163,1796,1797],{},"\"forward\""," and ",[163,1800,1801],{},"\"nearest\""," have no spreadsheet equivalent and are genuinely useful for matching a reading to the next scheduled time or the closest one either way.",[151,1804,1806],{"id":1805},"common-pitfalls-and-gotchas","Common pitfalls and gotchas",[1808,1809,1810,1826],"table",{},[1811,1812,1813],"thead",{},[1814,1815,1816,1820,1823],"tr",{},[1817,1818,1819],"th",{},"Symptom",[1817,1821,1822],{},"Cause",[1817,1824,1825],{},"Fix",[1827,1828,1829,1843,1856,1872,1886,1903,1923,1936],"tbody",{},[1814,1830,1831,1835,1838],{},[1832,1833,1834],"td",{},"Nothing matches at all",[1832,1836,1837],{},"Key is text in one file, numeric in the other",[1832,1839,1840,1841],{},"Read both with ",[163,1842,741],{},[1814,1844,1845,1848,1851],{},[1832,1846,1847],{},"Row count grew after the merge",[1832,1849,1850],{},"Duplicate keys in the lookup",[1832,1852,1853,1855],{},[163,1854,734],{},", then de-duplicate deliberately",[1814,1857,1858,1861,1867],{},[1832,1859,1860],{},"Rows disappeared",[1832,1862,1863,1866],{},[163,1864,1865],{},"how=\"inner\""," (the default)",[1832,1868,1869,1870],{},"Use ",[163,1871,723],{},[1814,1873,1874,1877,1880],{},[1832,1875,1876],{},"Whitespace stops a match",[1832,1878,1879],{},"Trailing spaces from the export",[1832,1881,1882,1885],{},[163,1883,1884],{},".str.strip()"," both keys first",[1814,1887,1888,1891,1900],{},[1832,1889,1890],{},"Case-sensitive misses",[1832,1892,1893,1896,1897],{},[163,1894,1895],{},"A-100"," versus ",[163,1898,1899],{},"a-100",[1832,1901,1902],{},"Normalise case on both sides",[1814,1904,1905,1914,1917],{},[1832,1906,1907,1798,1910,1913],{},[163,1908,1909],{},"_x",[163,1911,1912],{},"_y"," suffixes appear",[1832,1915,1916],{},"Both frames have a column of that name",[1832,1918,1919,1922],{},[163,1920,1921],{},"suffixes=",", or select the columns you want first",[1814,1924,1925,1930,1933],{},[1832,1926,1927,1929],{},[163,1928,1398],{}," raises about ordering",[1832,1931,1932],{},"Frames not sorted on the key",[1832,1934,1935],{},"Sort both before merging",[1814,1937,1938,1941,1944],{},[1832,1939,1940],{},"Totals silently low",[1832,1942,1943],{},"Missing prices filled with 0",[1832,1945,1946],{},"Fill labels, never fill money",[151,1948,1950],{"id":1949},"performance-and-scale-notes","Performance and scale notes",[10,1952,1953,1955,1956,1959],{},[163,1954,1024],{}," over a Series is the fastest option and allocates the least, so use it for single-column lookups on large frames. ",[163,1957,1958],{},"merge"," is a hash join and comfortably handles millions of rows, but it materialises the result — a many-to-many merge on a large frame is the usual cause of a report suddenly needing gigabytes.",[10,1961,1962,1963,1966,1967,1971],{},"When the same lookup table is used repeatedly, build the mapping once outside the loop rather than calling ",[163,1964,1965],{},"set_index"," on every iteration. And when the lookup lives in a database rather than a workbook, consider doing the join in SQL instead of transferring the whole table to pandas — ",[17,1968,1970],{"href":1969},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmoving-data-between-excel-and-databases\u002F","Moving Data Between Excel and Databases"," covers where to draw that line.",[151,1973,1975],{"id":1974},"conclusion","Conclusion",[10,1977,1978,1981,1982,1984,1985,1987,1988,1990,1991,1993,1994,1996],{},[163,1979,1980],{},"merge(how=\"left\")"," is the general VLOOKUP replacement and brings across as many columns as you need; ",[163,1983,1024],{}," is the faster answer when you need exactly one and cannot accidentally multiply rows; ",[163,1986,1398],{}," is the banded lookup VLOOKUP's fourth argument was doing all along. Read the key columns as strings so a type mismatch cannot silently break every match, pass ",[163,1989,914],{}," to get the unmatched rows as data rather than as ",[163,1992,516],{},", and let ",[163,1995,734],{}," fail loudly on the duplicate key that would otherwise inflate every total in the report.",[151,1998,2000],{"id":1999},"frequently-asked-questions","Frequently asked questions",[10,2002,2003,2012,2014,2015,2017,2018,2020],{},[2004,2005,2006,2007,926,2009,2011],"strong",{},"Is ",[163,2008,1958],{},[163,2010,1024],{}," the closer equivalent to VLOOKUP?",[163,2013,1958],{}," with ",[163,2016,723],{}," is the general answer and handles several returned columns. ",[163,2019,1024],{}," is closer in spirit for a single key-to-value translation and is faster, but it only returns one column.",[10,2022,2023,2026,2027,2029,2030,2032,2033,2036,2037,2039],{},[2004,2024,2025],{},"How do I find the rows that did not match?","\nPass ",[163,2028,914],{}," to ",[163,2031,1958],{}," and filter on ",[163,2034,2035],{},"_merge == \"left_only\"",". That is the equivalent of scanning for ",[163,2038,516],{},", except it gives you the rows rather than a marker.",[10,2041,2042,2045,2046,2048],{},[2004,2043,2044],{},"Why did my row count grow after the merge?","\nThe lookup table has duplicate keys, so each source row matched several. De-duplicate the lookup first, or use ",[163,2047,734],{}," to make pandas raise instead.",[10,2050,2051,2054,2057],{},[2004,2052,2053],{},"What replaces VLOOKUP's TRUE fourth argument?",[163,2055,2056],{},"pd.merge_asof",", which joins on the nearest key at or below the value — the banded lookup used for rate tables and for matching a reading to the most recent timestamp.",[151,2059,2061],{"id":2060},"related","Related",[10,2063,2064],{},"Up to the parent guide:",[2066,2067,2068],"ul",{},[2069,2070,2071,2073],"li",{},[17,2072,20],{"href":19}," — join types, keys and the wider merge vocabulary.",[10,2075,2076],{},"Related guides:",[2066,2078,2079,2086,2095,2102],{},[2069,2080,2081,2085],{},[17,2082,2084],{"href":2083},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fmerge-two-excel-files-on-common-column-python\u002F","Merge Two Excel Files on a Common Column in Python"," — the file-to-file version of this join.",[2069,2087,2088,2090,2091,2094],{},[17,2089,1384],{"href":1383}," — investigating the duplicate keys ",[163,2092,2093],{},"validate"," catches.",[2069,2096,2097,2101],{},[17,2098,2100],{"href":2099},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fcheck-excel-data-types-with-pandas\u002F","Check Excel Data Types with pandas"," — the type mismatch behind most failed matches.",[2069,2103,2104,2108],{},[17,2105,2107],{"href":2106},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffill-missing-values-in-excel-with-pandas-fillna\u002F","Fill Missing Values in Excel with pandas fillna"," — deciding what an unmatched row should become.",[2110,2111,2112],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .s-wDw, html code.shiki .s-wDw{--shiki-default:#6A737D;--shiki-dark:#BDC4CC}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":161,"searchDepth":215,"depth":215,"links":2114},[2115,2116,2117,2118,2119,2120,2121,2122,2123,2124,2125,2126],{"id":153,"depth":215,"text":154},{"id":188,"depth":215,"text":189},{"id":520,"depth":215,"text":521},{"id":749,"depth":215,"text":750},{"id":1017,"depth":215,"text":1018},{"id":1232,"depth":215,"text":1233},{"id":1388,"depth":215,"text":1389},{"id":1805,"depth":215,"text":1806},{"id":1949,"depth":215,"text":1950},{"id":1974,"depth":215,"text":1975},{"id":1999,"depth":215,"text":2000},{"id":2060,"depth":215,"text":2061},"2026-08-11","merge is the real answer, map is the fast one: translating a VLOOKUP into pandas, keeping the unmatched rows visible, handling duplicate lookup keys, and the approximate-match case VLOOKUP's fourth argument hides.","md",[2131,2134,2136,2138],{"q":2132,"a":2133},"Is merge or map the closer equivalent to VLOOKUP?","merge with how=\"left\" is the general answer and handles several returned columns. map is closer in spirit for a single key-to-value translation and is faster, but it only returns one column.",{"q":2025,"a":2135},"Pass indicator=True to merge and filter on _merge == \"left_only\". That is the equivalent of scanning for",{"q":2044,"a":2137},"The lookup table has duplicate keys, so each source row matched several. De-duplicate the lookup first, or use validate=\"m:1\" to make pandas raise instead.",{"q":2053,"a":2139},"pd.merge_asof, which joins on the nearest key at or below the value — the banded lookup used for rate tables and for matching a reading to the most recent timestamp.",{"breadcrumb":2141},[2142,2145,2148,2149],{"name":2143,"item":2144},"Home","\u002F",{"name":2146,"item":2147},"Advanced Data Transformation and Cleaning","\u002Fadvanced-data-transformation-and-cleaning\u002F",{"name":20,"item":19},{"name":5,"item":2150},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fvlookup-equivalent-in-pandas-for-excel-files\u002F","\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fvlookup-equivalent-in-pandas-for-excel-files",{"title":2153,"description":2154},"VLOOKUP in pandas — the Python Equivalent","Replace Excel VLOOKUP with pandas: merge with how='left', indicator to find misses, map for a simple key-value lookup, duplicate keys, and merge_asof for approximate matches.","vlookup-equivalent-in-pandas-for-excel-files","advanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fvlookup-equivalent-in-pandas-for-excel-files\u002Findex","how-to","LkXupBxFeRIaBcG5e-9KhPCcyfXiflkAk5uP3Dsqku8",[2160,2163],{"title":2084,"path":2161,"stem":2162,"children":-1},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fmerge-two-excel-files-on-common-column-python","advanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Fmerge-two-excel-files-on-common-column-python\u002Findex",{"title":1970,"path":2164,"stem":2165,"children":-1},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmoving-data-between-excel-and-databases","advanced-data-transformation-and-cleaning\u002Fmoving-data-between-excel-and-databases\u002Findex",1786800027161]