[{"data":1,"prerenderedAt":2467},["ShallowReactive",2],{"doc:\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-an-excel-file-from-a-url-or-bytes-in-python":3,"surround:\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-an-excel-file-from-a-url-or-bytes-in-python":2458},{"id":4,"title":5,"body":6,"dateModified":2435,"datePublished":2435,"description":2436,"extension":2437,"faq":2438,"meta":2449,"navigation":240,"path":2450,"seo":2451,"slug":2454,"stem":2455,"type":2456,"__hash__":2457},"docs\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-an-excel-file-from-a-url-or-bytes-in-python\u002Findex.md","Read an Excel File from a URL or Bytes in Python",{"type":7,"value":8,"toc":2418},"minimark",[9,19,162,167,198,205,209,212,260,263,267,586,609,613,616,717,728,732,735,894,901,990,994,1101,1111,1115,1118,1187,1190,1243,1246,1250,1253,1466,1472,1476,1479,1671,1678,1682,1689,2160,2167,2225,2229,2279,2283,2294,2298,2305,2309,2322,2336,2346,2362,2372,2376,2414],[10,11,12,13,18],"p",{},"Workbooks increasingly arrive over a network rather than sitting on a disk: a partner's reporting portal, an S3 object, a database blob, a file a user just uploaded. Every Python Excel reader accepts a file-like object, so none of that needs a temporary file — but reading straight from a response has failure modes a local read does not, starting with the login page that arrives instead of the spreadsheet. This guide reads a workbook from each source safely. It belongs to ",[14,15,17],"a",{"href":16},"\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002F","Handling Excel File Formats and Conversions",".",[20,21,29,30,29,34,29,38,29,45,29,52,29,61,29,67,29,70,29,74,29,77,29,81,29,83,29,87,29,95,29,100,29,103,29,106,29,110,29,113,29,117,29,120,29,128,29,134,29,139,29,145,29,149,29,153,29,158],"svg",{"viewBox":22,"role":23,"ariaLabelledBy":24,"xmlns":27,"style":28},"0 0 760 226","img",[25,26],"url-t","url-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:760px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[31,32,33],"title",{"id":25},"Sources that all end up as the same bytes",[35,36,37],"desc",{"id":26},"An HTTP response, an S3 object, a database blob and a web upload each produce a byte string, which is wrapped in BytesIO and read by the same reader.",[39,40],"rect",{"x":41,"y":41,"width":42,"height":43,"fill":44},"0","760","226","#ffffff",[46,47,51],"text",{"x":48,"y":49,"style":50},"380","26","font-size:13px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","Four sources, one reader",[39,53],{"x":54,"y":55,"width":56,"height":57,"rx":58,"fill":59,"stroke":60},"30","44","176","38","8","#ebebfd","var(--line,#cdd5e6)",[46,62,66],{"x":63,"y":64,"style":65},"118","68","font-size:12px;fill:var(--text,#172033);text-anchor:middle","HTTP response",[39,68],{"x":54,"y":69,"width":56,"height":57,"rx":58,"fill":59,"stroke":60},"88",[46,71,73],{"x":63,"y":72,"style":65},"112","S3 object body",[39,75],{"x":54,"y":76,"width":56,"height":57,"rx":58,"fill":59,"stroke":60},"132",[46,78,80],{"x":63,"y":79,"style":65},"156","database blob",[39,82],{"x":54,"y":56,"width":56,"height":57,"rx":58,"fill":59,"stroke":60},[46,84,86],{"x":63,"y":85,"style":65},"200","web upload",[88,89],"line",{"x1":90,"y1":91,"x2":92,"y2":72,"stroke":93,"style":94},"206","63","256","var(--brand,#5b5cf0)","stroke-width:2px",[96,97],"polygon",{"points":98,"fill":99},"256,112 244,109 249,101","#5b5cf0",[88,101],{"x1":90,"y1":102,"x2":92,"y2":63,"stroke":93,"style":94},"107",[96,104],{"points":105,"fill":99},"256,118 245,113 247,123",[88,107],{"x1":90,"y1":108,"x2":92,"y2":109,"stroke":93,"style":94},"151","130",[96,111],{"points":112,"fill":99},"256,130 246,136 250,126",[88,114],{"x1":90,"y1":115,"x2":92,"y2":116,"stroke":93,"style":94},"195","136",[96,118],{"points":119,"fill":99},"256,136 249,147 244,139",[39,121],{"x":122,"y":123,"width":124,"height":125,"rx":126,"fill":127,"stroke":60},"260","98","180","60","12","#fdefd8",[46,129,133],{"x":130,"y":131,"style":132},"350","124","font-size:12.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","bytes",[46,135,138],{"x":130,"y":136,"style":137},"146","font-size:11.5px;fill:var(--gold-ink,#7a4e06);text-anchor:middle","check before parsing",[88,140],{"x1":141,"y1":142,"x2":143,"y2":142,"stroke":144,"style":94},"440","128","486","var(--teal,#0f9488)",[96,146],{"points":147,"fill":148},"486,128 476,123 476,133","#0f766e",[39,150],{"x":151,"y":123,"width":124,"height":125,"rx":126,"fill":152,"stroke":60},"490","#d9f4f1",[46,154,157],{"x":155,"y":131,"style":156},"580","font-size:12.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","io.BytesIO",[46,159,161],{"x":155,"y":136,"style":160},"font-size:11.5px;fill:var(--teal-ink,#0b6157);text-anchor:middle","read_excel \u002F load_workbook",[163,164,166],"h2",{"id":165},"prerequisites","Prerequisites",[168,169,174],"pre",{"className":170,"code":171,"language":172,"meta":173,"style":173},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl requests\n","bash","",[175,176,177],"code",{"__ignoreMap":173},[178,179,181,185,189,192,195],"span",{"class":88,"line":180},1,[178,182,184],{"class":183},"sMTad","pip",[178,186,188],{"class":187},"srMev"," install",[178,190,191],{"class":187}," pandas",[178,193,194],{"class":187}," openpyxl",[178,196,197],{"class":187}," requests\n",[10,199,200,201,204],{},"Add ",[175,202,203],{},"boto3"," for the S3 section.",[163,206,208],{"id":207},"the-one-liner-and-why-to-outgrow-it","The one-liner, and why to outgrow it",[10,210,211],{},"pandas will fetch a URL itself:",[168,213,217],{"className":214,"code":215,"language":216,"meta":173,"style":173},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\ndf = pd.read_excel(\"https:\u002F\u002Fexample.com\u002Freports\u002Flatest.xlsx\")\n","python",[175,218,219,235,242],{"__ignoreMap":173},[178,220,221,225,229,232],{"class":88,"line":180},[178,222,224],{"class":223},"s-kum","import",[178,226,228],{"class":227},"skGVy"," pandas ",[178,230,231],{"class":223},"as",[178,233,234],{"class":227}," pd\n",[178,236,238],{"class":88,"line":237},2,[178,239,241],{"emptyLinePlaceholder":240},true,"\n",[178,243,245,248,251,254,257],{"class":88,"line":244},3,[178,246,247],{"class":227},"df ",[178,249,250],{"class":223},"=",[178,252,253],{"class":227}," pd.read_excel(",[178,255,256],{"class":187},"\"https:\u002F\u002Fexample.com\u002Freports\u002Flatest.xlsx\"",[178,258,259],{"class":227},")\n",[10,261,262],{},"That is fine for a public file in a notebook. In a scheduled job it is a liability: no timeout, no retry, no authentication header, and no way to notice that the server returned a 200 with an HTML login page. Fetch deliberately instead.",[163,264,266],{"id":265},"fetch-verify-then-read","Fetch, verify, then read",[168,268,270],{"className":214,"code":269,"language":216,"meta":173,"style":173},"\"\"\"Download a workbook and read it, refusing anything that is not one.\"\"\"\nimport io\n\nimport pandas as pd\nimport requests\n\ndef read_excel_url(url: str, **kwargs) -> pd.DataFrame:\n    resp = requests.get(url, timeout=30, headers={\"Accept\": \"*\u002F*\"})\n    resp.raise_for_status()\n\n    content_type = resp.headers.get(\"Content-Type\", \"\")\n    if \"html\" in content_type:\n        raise ValueError(f\"{url} returned HTML — probably a login or error page\")\n    if not resp.content[:2] == b\"PK\":\n        raise ValueError(f\"{url} is not an .xlsx (starts with {resp.content[:8]!r})\")\n\n    return pd.read_excel(io.BytesIO(resp.content), engine=\"openpyxl\", **kwargs)\n\ndf = read_excel_url(\"https:\u002F\u002Fexample.com\u002Freports\u002Flatest.xlsx\", sheet_name=\"Q3\")\n",[175,271,272,277,284,288,299,306,311,337,378,384,389,410,425,457,486,528,533,557,562],{"__ignoreMap":173},[178,273,274],{"class":88,"line":180},[178,275,276],{"class":187},"\"\"\"Download a workbook and read it, refusing anything that is not one.\"\"\"\n",[178,278,279,281],{"class":88,"line":237},[178,280,224],{"class":223},[178,282,283],{"class":227}," io\n",[178,285,286],{"class":88,"line":244},[178,287,241],{"emptyLinePlaceholder":240},[178,289,291,293,295,297],{"class":88,"line":290},4,[178,292,224],{"class":223},[178,294,228],{"class":227},[178,296,231],{"class":223},[178,298,234],{"class":227},[178,300,302,304],{"class":88,"line":301},5,[178,303,224],{"class":223},[178,305,197],{"class":227},[178,307,309],{"class":88,"line":308},6,[178,310,241],{"emptyLinePlaceholder":240},[178,312,314,317,321,324,328,331,334],{"class":88,"line":313},7,[178,315,316],{"class":223},"def",[178,318,320],{"class":319},"s_Opv"," read_excel_url",[178,322,323],{"class":227},"(url: ",[178,325,327],{"class":326},"sP0c6","str",[178,329,330],{"class":227},", ",[178,332,333],{"class":223},"**",[178,335,336],{"class":227},"kwargs) -> pd.DataFrame:\n",[178,338,340,343,345,348,352,354,356,358,361,363,366,369,372,375],{"class":88,"line":339},8,[178,341,342],{"class":227},"    resp ",[178,344,250],{"class":223},[178,346,347],{"class":227}," requests.get(url, ",[178,349,351],{"class":350},"sa561","timeout",[178,353,250],{"class":223},[178,355,54],{"class":326},[178,357,330],{"class":227},[178,359,360],{"class":350},"headers",[178,362,250],{"class":223},[178,364,365],{"class":227},"{",[178,367,368],{"class":187},"\"Accept\"",[178,370,371],{"class":227},": ",[178,373,374],{"class":187},"\"*\u002F*\"",[178,376,377],{"class":227},"})\n",[178,379,381],{"class":88,"line":380},9,[178,382,383],{"class":227},"    resp.raise_for_status()\n",[178,385,387],{"class":88,"line":386},10,[178,388,241],{"emptyLinePlaceholder":240},[178,390,392,395,397,400,403,405,408],{"class":88,"line":391},11,[178,393,394],{"class":227},"    content_type ",[178,396,250],{"class":223},[178,398,399],{"class":227}," resp.headers.get(",[178,401,402],{"class":187},"\"Content-Type\"",[178,404,330],{"class":227},[178,406,407],{"class":187},"\"\"",[178,409,259],{"class":227},[178,411,413,416,419,422],{"class":88,"line":412},12,[178,414,415],{"class":223},"    if",[178,417,418],{"class":187}," \"html\"",[178,420,421],{"class":223}," in",[178,423,424],{"class":227}," content_type:\n",[178,426,428,431,434,437,440,443,446,449,452,455],{"class":88,"line":427},13,[178,429,430],{"class":223},"        raise",[178,432,433],{"class":326}," ValueError",[178,435,436],{"class":227},"(",[178,438,439],{"class":223},"f",[178,441,442],{"class":187},"\"",[178,444,365],{"class":445},"sSjpA",[178,447,448],{"class":227},"url",[178,450,451],{"class":445},"}",[178,453,454],{"class":187}," returned HTML — probably a login or error page\"",[178,456,259],{"class":227},[178,458,460,462,465,468,471,474,477,480,483],{"class":88,"line":459},14,[178,461,415],{"class":223},[178,463,464],{"class":223}," not",[178,466,467],{"class":227}," resp.content[:",[178,469,470],{"class":326},"2",[178,472,473],{"class":227},"] ",[178,475,476],{"class":223},"==",[178,478,479],{"class":223}," b",[178,481,482],{"class":187},"\"PK\"",[178,484,485],{"class":227},":\n",[178,487,489,491,493,495,497,499,501,503,505,508,510,513,515,518,521,523,526],{"class":88,"line":488},15,[178,490,430],{"class":223},[178,492,433],{"class":326},[178,494,436],{"class":227},[178,496,439],{"class":223},[178,498,442],{"class":187},[178,500,365],{"class":445},[178,502,448],{"class":227},[178,504,451],{"class":445},[178,506,507],{"class":187}," is not an .xlsx (starts with ",[178,509,365],{"class":445},[178,511,512],{"class":227},"resp.content[:",[178,514,58],{"class":326},[178,516,517],{"class":227},"]",[178,519,520],{"class":223},"!r",[178,522,451],{"class":445},[178,524,525],{"class":187},")\"",[178,527,259],{"class":227},[178,529,531],{"class":88,"line":530},16,[178,532,241],{"emptyLinePlaceholder":240},[178,534,536,539,542,545,547,550,552,554],{"class":88,"line":535},17,[178,537,538],{"class":223},"    return",[178,540,541],{"class":227}," pd.read_excel(io.BytesIO(resp.content), ",[178,543,544],{"class":350},"engine",[178,546,250],{"class":223},[178,548,549],{"class":187},"\"openpyxl\"",[178,551,330],{"class":227},[178,553,333],{"class":223},[178,555,556],{"class":227},"kwargs)\n",[178,558,560],{"class":88,"line":559},18,[178,561,241],{"emptyLinePlaceholder":240},[178,563,565,567,569,572,574,576,579,581,584],{"class":88,"line":564},19,[178,566,247],{"class":227},[178,568,250],{"class":223},[178,570,571],{"class":227}," read_excel_url(",[178,573,256],{"class":187},[178,575,330],{"class":227},[178,577,578],{"class":350},"sheet_name",[178,580,250],{"class":223},[178,582,583],{"class":187},"\"Q3\"",[178,585,259],{"class":227},[10,587,588,589,592,593,596,597,600,601,604,605,18],{},"The magic-byte check is the important one. ",[175,590,591],{},"Content-Type"," is often wrong — plenty of servers label an ",[175,594,595],{},".xlsx"," as ",[175,598,599],{},"application\u002Foctet-stream"," — but the first two bytes never lie: a modern workbook is a zip, so it starts with ",[175,602,603],{},"PK",". The whole family of failures is catalogued in ",[14,606,608],{"href":607},"\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-badzipfile-error-when-reading-excel-in-python\u002F","Fix BadZipFile when reading an Excel file in Python",[163,610,612],{"id":611},"authenticate","Authenticate",[10,614,615],{},"Most real sources need credentials, and they belong in the environment, not the code:",[168,617,619],{"className":214,"code":618,"language":216,"meta":173,"style":173},"import os\n\nimport requests\n\nsession = requests.Session()\nsession.headers[\"Authorization\"] = f\"Bearer {os.environ['REPORT_API_TOKEN']}\"\n\nresp = session.get(\"https:\u002F\u002Fapi.example.com\u002Fexports\u002Fmonthly.xlsx\", timeout=30)\nresp.raise_for_status()\n",[175,620,621,628,632,638,642,652,685,689,712],{"__ignoreMap":173},[178,622,623,625],{"class":88,"line":180},[178,624,224],{"class":223},[178,626,627],{"class":227}," os\n",[178,629,630],{"class":88,"line":237},[178,631,241],{"emptyLinePlaceholder":240},[178,633,634,636],{"class":88,"line":244},[178,635,224],{"class":223},[178,637,197],{"class":227},[178,639,640],{"class":88,"line":290},[178,641,241],{"emptyLinePlaceholder":240},[178,643,644,647,649],{"class":88,"line":301},[178,645,646],{"class":227},"session ",[178,648,250],{"class":223},[178,650,651],{"class":227}," requests.Session()\n",[178,653,654,657,660,662,664,667,670,672,675,678,680,682],{"class":88,"line":308},[178,655,656],{"class":227},"session.headers[",[178,658,659],{"class":187},"\"Authorization\"",[178,661,473],{"class":227},[178,663,250],{"class":223},[178,665,666],{"class":223}," f",[178,668,669],{"class":187},"\"Bearer ",[178,671,365],{"class":445},[178,673,674],{"class":227},"os.environ[",[178,676,677],{"class":187},"'REPORT_API_TOKEN'",[178,679,517],{"class":227},[178,681,451],{"class":445},[178,683,684],{"class":187},"\"\n",[178,686,687],{"class":88,"line":313},[178,688,241],{"emptyLinePlaceholder":240},[178,690,691,694,696,699,702,704,706,708,710],{"class":88,"line":339},[178,692,693],{"class":227},"resp ",[178,695,250],{"class":223},[178,697,698],{"class":227}," session.get(",[178,700,701],{"class":187},"\"https:\u002F\u002Fapi.example.com\u002Fexports\u002Fmonthly.xlsx\"",[178,703,330],{"class":227},[178,705,351],{"class":350},[178,707,250],{"class":223},[178,709,54],{"class":326},[178,711,259],{"class":227},[178,713,714],{"class":88,"line":380},[178,715,716],{"class":227},"resp.raise_for_status()\n",[10,718,719,720,723,724,727],{},"Basic auth is ",[175,721,722],{},"auth=(user, password)","; a cookie-based portal usually means posting to a login endpoint first and reusing the session. Using one ",[175,725,726],{},"Session"," matters beyond credentials: it reuses the TCP connection, which is a real saving when fetching many files.",[163,729,731],{"id":730},"retry-transient-failures","Retry transient failures",[10,733,734],{},"Report portals fail intermittently, and a nightly job should not.",[168,736,738],{"className":214,"code":737,"language":216,"meta":173,"style":173},"from requests.adapters import HTTPAdapter\nfrom urllib3.util.retry import Retry\n\nretry = Retry(\n    total=4,\n    backoff_factor=1.5,                     # 0s, 1.5s, 3s, 6s\n    status_forcelist=(429, 500, 502, 503, 504),\n    allowed_methods=frozenset([\"GET\"]),\n)\nsession = requests.Session()\nsession.mount(\"https:\u002F\u002F\", HTTPAdapter(max_retries=retry))\n",[175,739,740,753,765,769,779,792,809,844,863,867,875],{"__ignoreMap":173},[178,741,742,745,748,750],{"class":88,"line":180},[178,743,744],{"class":223},"from",[178,746,747],{"class":227}," requests.adapters ",[178,749,224],{"class":223},[178,751,752],{"class":227}," HTTPAdapter\n",[178,754,755,757,760,762],{"class":88,"line":237},[178,756,744],{"class":223},[178,758,759],{"class":227}," urllib3.util.retry ",[178,761,224],{"class":223},[178,763,764],{"class":227}," Retry\n",[178,766,767],{"class":88,"line":244},[178,768,241],{"emptyLinePlaceholder":240},[178,770,771,774,776],{"class":88,"line":290},[178,772,773],{"class":227},"retry ",[178,775,250],{"class":223},[178,777,778],{"class":227}," Retry(\n",[178,780,781,784,786,789],{"class":88,"line":301},[178,782,783],{"class":350},"    total",[178,785,250],{"class":223},[178,787,788],{"class":326},"4",[178,790,791],{"class":227},",\n",[178,793,794,797,799,802,805],{"class":88,"line":308},[178,795,796],{"class":350},"    backoff_factor",[178,798,250],{"class":223},[178,800,801],{"class":326},"1.5",[178,803,804],{"class":227},",                     ",[178,806,808],{"class":807},"s-wDw","# 0s, 1.5s, 3s, 6s\n",[178,810,811,814,816,818,821,823,826,828,831,833,836,838,841],{"class":88,"line":313},[178,812,813],{"class":350},"    status_forcelist",[178,815,250],{"class":223},[178,817,436],{"class":227},[178,819,820],{"class":326},"429",[178,822,330],{"class":227},[178,824,825],{"class":326},"500",[178,827,330],{"class":227},[178,829,830],{"class":326},"502",[178,832,330],{"class":227},[178,834,835],{"class":326},"503",[178,837,330],{"class":227},[178,839,840],{"class":326},"504",[178,842,843],{"class":227},"),\n",[178,845,846,849,851,854,857,860],{"class":88,"line":339},[178,847,848],{"class":350},"    allowed_methods",[178,850,250],{"class":223},[178,852,853],{"class":326},"frozenset",[178,855,856],{"class":227},"([",[178,858,859],{"class":187},"\"GET\"",[178,861,862],{"class":227},"]),\n",[178,864,865],{"class":88,"line":380},[178,866,259],{"class":227},[178,868,869,871,873],{"class":88,"line":386},[178,870,646],{"class":227},[178,872,250],{"class":223},[178,874,651],{"class":227},[178,876,877,880,883,886,889,891],{"class":88,"line":391},[178,878,879],{"class":227},"session.mount(",[178,881,882],{"class":187},"\"https:\u002F\u002F\"",[178,884,885],{"class":227},", HTTPAdapter(",[178,887,888],{"class":350},"max_retries",[178,890,250],{"class":223},[178,892,893],{"class":227},"retry))\n",[10,895,896,897,18],{},"Retrying only idempotent methods and only on genuinely transient statuses is what keeps this from hiding a real problem — a 404 or a 403 fails immediately, as it should. The job-level equivalent is in ",[14,898,900],{"href":899},"\u002Fautomating-reporting-workflows\u002Ferror-handling-and-logging-in-excel-automation\u002Fretry-a-failed-excel-report-job-in-python\u002F","Retry a failed Excel report job in Python",[20,902,29,907,29,910,29,913,29,916,29,919,29,926,29,931,29,936,29,939,29,941,29,947,29,951,29,955,29,958,29,961,29,965,29,967,29,971,29,974,29,977,29,982,29,986],{"viewBox":903,"role":23,"ariaLabelledBy":904,"xmlns":27,"style":28},"0 0 760 214",[905,906],"url2-t","url2-d",[31,908,909],{"id":905},"Checks between the download and the parse",[35,911,912],{"id":906},"A response passes a status check, a content-type check and a magic-byte check before it is parsed, so an error page never reaches the Excel reader.",[39,914],{"x":41,"y":41,"width":42,"height":915,"fill":44},"214",[46,917,918],{"x":48,"y":49,"style":50},"Three gates, each one line",[39,920],{"x":921,"y":922,"width":923,"height":125,"rx":924,"fill":925,"stroke":60},"24","76","140","11","#f0f2f5",[46,927,930],{"x":928,"y":72,"style":929},"94","font-size:12px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","response",[88,932],{"x1":933,"y1":934,"x2":935,"y2":934,"stroke":93,"style":94},"164","106","196",[96,937],{"points":938,"fill":99},"196,106 186,101 186,111",[39,940],{"x":85,"y":922,"width":923,"height":125,"rx":924,"fill":59,"stroke":60},[46,942,946],{"x":943,"y":944,"style":945},"270","104","font-size:12px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","status 2xx?",[46,948,950],{"x":943,"y":131,"style":949},"font-size:11px;fill:var(--muted,#5b6780);text-anchor:middle","raise_for_status",[88,952],{"x1":953,"y1":934,"x2":954,"y2":934,"stroke":93,"style":94},"340","372",[96,956],{"points":957,"fill":99},"372,106 362,101 362,111",[39,959],{"x":960,"y":922,"width":923,"height":125,"rx":924,"fill":59,"stroke":60},"376",[46,962,964],{"x":963,"y":944,"style":945},"446","not HTML?",[46,966,591],{"x":963,"y":131,"style":949},[88,968],{"x1":969,"y1":934,"x2":970,"y2":934,"stroke":93,"style":94},"516","548",[96,972],{"points":973,"fill":99},"548,106 538,101 538,111",[39,975],{"x":976,"y":922,"width":923,"height":125,"rx":924,"fill":152,"stroke":60},"552",[46,978,981],{"x":979,"y":944,"style":980},"622","font-size:12px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","starts with PK?",[46,983,985],{"x":979,"y":131,"style":984},"font-size:11px;fill:var(--teal-ink,#0b6157);text-anchor:middle","then parse",[46,987,989],{"x":48,"y":124,"style":988},"font-size:11.5px;fill:var(--muted,#5b6780);text-anchor:middle","A 200 response is not evidence that the body is a workbook",[163,991,993],{"id":992},"read-from-s3-and-other-object-storage","Read from S3 and other object storage",[168,995,997],{"className":214,"code":996,"language":216,"meta":173,"style":173},"import io\n\nimport boto3\nimport pandas as pd\n\ns3 = boto3.client(\"s3\")\nbuffer = io.BytesIO()\ns3.download_fileobj(\"reports-bucket\", \"monthly\u002Fsales.xlsx\", buffer)\nbuffer.seek(0)\n\ndf = pd.read_excel(buffer, engine=\"openpyxl\")\n",[175,998,999,1005,1009,1016,1026,1030,1045,1055,1071,1080,1084],{"__ignoreMap":173},[178,1000,1001,1003],{"class":88,"line":180},[178,1002,224],{"class":223},[178,1004,283],{"class":227},[178,1006,1007],{"class":88,"line":237},[178,1008,241],{"emptyLinePlaceholder":240},[178,1010,1011,1013],{"class":88,"line":244},[178,1012,224],{"class":223},[178,1014,1015],{"class":227}," boto3\n",[178,1017,1018,1020,1022,1024],{"class":88,"line":290},[178,1019,224],{"class":223},[178,1021,228],{"class":227},[178,1023,231],{"class":223},[178,1025,234],{"class":227},[178,1027,1028],{"class":88,"line":301},[178,1029,241],{"emptyLinePlaceholder":240},[178,1031,1032,1035,1037,1040,1043],{"class":88,"line":308},[178,1033,1034],{"class":227},"s3 ",[178,1036,250],{"class":223},[178,1038,1039],{"class":227}," boto3.client(",[178,1041,1042],{"class":187},"\"s3\"",[178,1044,259],{"class":227},[178,1046,1047,1050,1052],{"class":88,"line":313},[178,1048,1049],{"class":227},"buffer ",[178,1051,250],{"class":223},[178,1053,1054],{"class":227}," io.BytesIO()\n",[178,1056,1057,1060,1063,1065,1068],{"class":88,"line":339},[178,1058,1059],{"class":227},"s3.download_fileobj(",[178,1061,1062],{"class":187},"\"reports-bucket\"",[178,1064,330],{"class":227},[178,1066,1067],{"class":187},"\"monthly\u002Fsales.xlsx\"",[178,1069,1070],{"class":227},", buffer)\n",[178,1072,1073,1076,1078],{"class":88,"line":380},[178,1074,1075],{"class":227},"buffer.seek(",[178,1077,41],{"class":326},[178,1079,259],{"class":227},[178,1081,1082],{"class":88,"line":386},[178,1083,241],{"emptyLinePlaceholder":240},[178,1085,1086,1088,1090,1093,1095,1097,1099],{"class":88,"line":391},[178,1087,247],{"class":227},[178,1089,250],{"class":223},[178,1091,1092],{"class":227}," pd.read_excel(buffer, ",[178,1094,544],{"class":350},[178,1096,250],{"class":223},[178,1098,549],{"class":187},[178,1100,259],{"class":227},[10,1102,1103,1104,1107,1108,1110],{},"Do not hand the streaming body from ",[175,1105,1106],{},"get_object"," straight to the reader. An ",[175,1109,595],{}," is a zip, and a zip is read by seeking to its directory at the end of the file — a forward-only stream cannot do that. Download into a buffer first, exactly as above.",[163,1112,1114],{"id":1113},"read-an-upload-or-a-database-blob","Read an upload or a database blob",[10,1116,1117],{},"A web upload is already a file-like object, so it can be read directly:",[168,1119,1121],{"className":214,"code":1120,"language":216,"meta":173,"style":173},"# Flask\ndf = pd.read_excel(request.files[\"report\"], engine=\"openpyxl\")\n\n# Django\ndf = pd.read_excel(request.FILES[\"report\"], engine=\"openpyxl\")\n",[175,1122,1123,1128,1151,1155,1160],{"__ignoreMap":173},[178,1124,1125],{"class":88,"line":180},[178,1126,1127],{"class":807},"# Flask\n",[178,1129,1130,1132,1134,1137,1140,1143,1145,1147,1149],{"class":88,"line":237},[178,1131,247],{"class":227},[178,1133,250],{"class":223},[178,1135,1136],{"class":227}," pd.read_excel(request.files[",[178,1138,1139],{"class":187},"\"report\"",[178,1141,1142],{"class":227},"], ",[178,1144,544],{"class":350},[178,1146,250],{"class":223},[178,1148,549],{"class":187},[178,1150,259],{"class":227},[178,1152,1153],{"class":88,"line":244},[178,1154,241],{"emptyLinePlaceholder":240},[178,1156,1157],{"class":88,"line":290},[178,1158,1159],{"class":807},"# Django\n",[178,1161,1162,1164,1166,1169,1172,1175,1177,1179,1181,1183,1185],{"class":88,"line":301},[178,1163,247],{"class":227},[178,1165,250],{"class":223},[178,1167,1168],{"class":227}," pd.read_excel(request.",[178,1170,1171],{"class":326},"FILES",[178,1173,1174],{"class":227},"[",[178,1176,1139],{"class":187},[178,1178,1142],{"class":227},[178,1180,544],{"class":350},[178,1182,250],{"class":223},[178,1184,549],{"class":187},[178,1186,259],{"class":227},[10,1188,1189],{},"A blob column is bytes, so wrap it:",[168,1191,1193],{"className":214,"code":1192,"language":216,"meta":173,"style":173},"import io\n\nrow = cursor.execute(\"SELECT content FROM uploads WHERE id = ?\", (upload_id,)).fetchone()\ndf = pd.read_excel(io.BytesIO(row[0]), engine=\"openpyxl\")\n",[175,1194,1195,1201,1205,1221],{"__ignoreMap":173},[178,1196,1197,1199],{"class":88,"line":180},[178,1198,224],{"class":223},[178,1200,283],{"class":227},[178,1202,1203],{"class":88,"line":237},[178,1204,241],{"emptyLinePlaceholder":240},[178,1206,1207,1210,1212,1215,1218],{"class":88,"line":244},[178,1208,1209],{"class":227},"row ",[178,1211,250],{"class":223},[178,1213,1214],{"class":227}," cursor.execute(",[178,1216,1217],{"class":187},"\"SELECT content FROM uploads WHERE id = ?\"",[178,1219,1220],{"class":227},", (upload_id,)).fetchone()\n",[178,1222,1223,1225,1227,1230,1232,1235,1237,1239,1241],{"class":88,"line":290},[178,1224,247],{"class":227},[178,1226,250],{"class":223},[178,1228,1229],{"class":227}," pd.read_excel(io.BytesIO(row[",[178,1231,41],{"class":326},[178,1233,1234],{"class":227},"]), ",[178,1236,544],{"class":350},[178,1238,250],{"class":223},[178,1240,549],{"class":187},[178,1242,259],{"class":227},[10,1244,1245],{},"In both cases validate before parsing — an upload is untrusted input, and the same magic-byte check applies.",[163,1247,1249],{"id":1248},"stream-a-large-download-to-disk","Stream a large download to disk",[10,1251,1252],{},"Holding a large file in memory and then parsing it means paying twice. Stream it to a temporary path instead:",[168,1254,1256],{"className":214,"code":1255,"language":216,"meta":173,"style":173},"import tempfile\nfrom pathlib import Path\n\nimport pandas as pd\nimport requests\n\nwith requests.get(url, stream=True, timeout=60) as resp:\n    resp.raise_for_status()\n    with tempfile.NamedTemporaryFile(suffix=\".xlsx\", delete=False) as tmp:\n        for chunk in resp.iter_content(chunk_size=1 \u003C\u003C 20):\n            tmp.write(chunk)\n        path = Path(tmp.name)\n\ntry:\n    df = pd.read_excel(path, engine=\"calamine\")\nfinally:\n    path.unlink(missing_ok=True)\n",[175,1257,1258,1265,1277,1281,1291,1297,1301,1332,1336,1369,1400,1405,1415,1419,1426,1445,1452],{"__ignoreMap":173},[178,1259,1260,1262],{"class":88,"line":180},[178,1261,224],{"class":223},[178,1263,1264],{"class":227}," tempfile\n",[178,1266,1267,1269,1272,1274],{"class":88,"line":237},[178,1268,744],{"class":223},[178,1270,1271],{"class":227}," pathlib ",[178,1273,224],{"class":223},[178,1275,1276],{"class":227}," Path\n",[178,1278,1279],{"class":88,"line":244},[178,1280,241],{"emptyLinePlaceholder":240},[178,1282,1283,1285,1287,1289],{"class":88,"line":290},[178,1284,224],{"class":223},[178,1286,228],{"class":227},[178,1288,231],{"class":223},[178,1290,234],{"class":227},[178,1292,1293,1295],{"class":88,"line":301},[178,1294,224],{"class":223},[178,1296,197],{"class":227},[178,1298,1299],{"class":88,"line":308},[178,1300,241],{"emptyLinePlaceholder":240},[178,1302,1303,1306,1308,1311,1313,1316,1318,1320,1322,1324,1327,1329],{"class":88,"line":313},[178,1304,1305],{"class":223},"with",[178,1307,347],{"class":227},[178,1309,1310],{"class":350},"stream",[178,1312,250],{"class":223},[178,1314,1315],{"class":326},"True",[178,1317,330],{"class":227},[178,1319,351],{"class":350},[178,1321,250],{"class":223},[178,1323,125],{"class":326},[178,1325,1326],{"class":227},") ",[178,1328,231],{"class":223},[178,1330,1331],{"class":227}," resp:\n",[178,1333,1334],{"class":88,"line":339},[178,1335,383],{"class":227},[178,1337,1338,1341,1344,1347,1349,1352,1354,1357,1359,1362,1364,1366],{"class":88,"line":380},[178,1339,1340],{"class":223},"    with",[178,1342,1343],{"class":227}," tempfile.NamedTemporaryFile(",[178,1345,1346],{"class":350},"suffix",[178,1348,250],{"class":223},[178,1350,1351],{"class":187},"\".xlsx\"",[178,1353,330],{"class":227},[178,1355,1356],{"class":350},"delete",[178,1358,250],{"class":223},[178,1360,1361],{"class":326},"False",[178,1363,1326],{"class":227},[178,1365,231],{"class":223},[178,1367,1368],{"class":227}," tmp:\n",[178,1370,1371,1374,1377,1380,1383,1386,1388,1391,1394,1397],{"class":88,"line":386},[178,1372,1373],{"class":223},"        for",[178,1375,1376],{"class":227}," chunk ",[178,1378,1379],{"class":223},"in",[178,1381,1382],{"class":227}," resp.iter_content(",[178,1384,1385],{"class":350},"chunk_size",[178,1387,250],{"class":223},[178,1389,1390],{"class":326},"1",[178,1392,1393],{"class":223}," \u003C\u003C",[178,1395,1396],{"class":326}," 20",[178,1398,1399],{"class":227},"):\n",[178,1401,1402],{"class":88,"line":391},[178,1403,1404],{"class":227},"            tmp.write(chunk)\n",[178,1406,1407,1410,1412],{"class":88,"line":412},[178,1408,1409],{"class":227},"        path ",[178,1411,250],{"class":223},[178,1413,1414],{"class":227}," Path(tmp.name)\n",[178,1416,1417],{"class":88,"line":427},[178,1418,241],{"emptyLinePlaceholder":240},[178,1420,1421,1424],{"class":88,"line":459},[178,1422,1423],{"class":223},"try",[178,1425,485],{"class":227},[178,1427,1428,1431,1433,1436,1438,1440,1443],{"class":88,"line":488},[178,1429,1430],{"class":227},"    df ",[178,1432,250],{"class":223},[178,1434,1435],{"class":227}," pd.read_excel(path, ",[178,1437,544],{"class":350},[178,1439,250],{"class":223},[178,1441,1442],{"class":187},"\"calamine\"",[178,1444,259],{"class":227},[178,1446,1447,1450],{"class":88,"line":530},[178,1448,1449],{"class":223},"finally",[178,1451,485],{"class":227},[178,1453,1454,1457,1460,1462,1464],{"class":88,"line":535},[178,1455,1456],{"class":227},"    path.unlink(",[178,1458,1459],{"class":350},"missing_ok",[178,1461,250],{"class":223},[178,1463,1315],{"class":326},[178,1465,259],{"class":227},[10,1467,1468,1469,1471],{},"The ",[175,1470,1449],{}," matters: a temporary file that outlives a crashed job fills a disk over weeks.",[163,1473,1475],{"id":1474},"cache-what-you-fetch","Cache what you fetch",[10,1477,1478],{},"If the same file is fetched repeatedly, use HTTP's own caching rather than re-downloading:",[168,1480,1482],{"className":214,"code":1481,"language":216,"meta":173,"style":173},"from pathlib import Path\n\nimport requests\n\ndef fetch_cached(url: str, cache: Path) -> bytes:\n    headers = {}\n    etag_file = cache.with_suffix(\".etag\")\n    if cache.exists() and etag_file.exists():\n        headers[\"If-None-Match\"] = etag_file.read_text()\n\n    resp = requests.get(url, headers=headers, timeout=30)\n    if resp.status_code == 304:\n        return cache.read_bytes()\n\n    resp.raise_for_status()\n    cache.write_bytes(resp.content)\n    if \"ETag\" in resp.headers:\n        etag_file.write_text(resp.headers[\"ETag\"])\n    return resp.content\n",[175,1483,1484,1494,1498,1504,1508,1526,1536,1551,1564,1579,1583,1606,1620,1628,1632,1636,1641,1653,1664],{"__ignoreMap":173},[178,1485,1486,1488,1490,1492],{"class":88,"line":180},[178,1487,744],{"class":223},[178,1489,1271],{"class":227},[178,1491,224],{"class":223},[178,1493,1276],{"class":227},[178,1495,1496],{"class":88,"line":237},[178,1497,241],{"emptyLinePlaceholder":240},[178,1499,1500,1502],{"class":88,"line":244},[178,1501,224],{"class":223},[178,1503,197],{"class":227},[178,1505,1506],{"class":88,"line":290},[178,1507,241],{"emptyLinePlaceholder":240},[178,1509,1510,1512,1515,1517,1519,1522,1524],{"class":88,"line":301},[178,1511,316],{"class":223},[178,1513,1514],{"class":319}," fetch_cached",[178,1516,323],{"class":227},[178,1518,327],{"class":326},[178,1520,1521],{"class":227},", cache: Path) -> ",[178,1523,133],{"class":326},[178,1525,485],{"class":227},[178,1527,1528,1531,1533],{"class":88,"line":308},[178,1529,1530],{"class":227},"    headers ",[178,1532,250],{"class":223},[178,1534,1535],{"class":227}," {}\n",[178,1537,1538,1541,1543,1546,1549],{"class":88,"line":313},[178,1539,1540],{"class":227},"    etag_file ",[178,1542,250],{"class":223},[178,1544,1545],{"class":227}," cache.with_suffix(",[178,1547,1548],{"class":187},"\".etag\"",[178,1550,259],{"class":227},[178,1552,1553,1555,1558,1561],{"class":88,"line":339},[178,1554,415],{"class":223},[178,1556,1557],{"class":227}," cache.exists() ",[178,1559,1560],{"class":223},"and",[178,1562,1563],{"class":227}," etag_file.exists():\n",[178,1565,1566,1569,1572,1574,1576],{"class":88,"line":380},[178,1567,1568],{"class":227},"        headers[",[178,1570,1571],{"class":187},"\"If-None-Match\"",[178,1573,473],{"class":227},[178,1575,250],{"class":223},[178,1577,1578],{"class":227}," etag_file.read_text()\n",[178,1580,1581],{"class":88,"line":386},[178,1582,241],{"emptyLinePlaceholder":240},[178,1584,1585,1587,1589,1591,1593,1595,1598,1600,1602,1604],{"class":88,"line":391},[178,1586,342],{"class":227},[178,1588,250],{"class":223},[178,1590,347],{"class":227},[178,1592,360],{"class":350},[178,1594,250],{"class":223},[178,1596,1597],{"class":227},"headers, ",[178,1599,351],{"class":350},[178,1601,250],{"class":223},[178,1603,54],{"class":326},[178,1605,259],{"class":227},[178,1607,1608,1610,1613,1615,1618],{"class":88,"line":412},[178,1609,415],{"class":223},[178,1611,1612],{"class":227}," resp.status_code ",[178,1614,476],{"class":223},[178,1616,1617],{"class":326}," 304",[178,1619,485],{"class":227},[178,1621,1622,1625],{"class":88,"line":427},[178,1623,1624],{"class":223},"        return",[178,1626,1627],{"class":227}," cache.read_bytes()\n",[178,1629,1630],{"class":88,"line":459},[178,1631,241],{"emptyLinePlaceholder":240},[178,1633,1634],{"class":88,"line":488},[178,1635,383],{"class":227},[178,1637,1638],{"class":88,"line":530},[178,1639,1640],{"class":227},"    cache.write_bytes(resp.content)\n",[178,1642,1643,1645,1648,1650],{"class":88,"line":535},[178,1644,415],{"class":223},[178,1646,1647],{"class":187}," \"ETag\"",[178,1649,421],{"class":223},[178,1651,1652],{"class":227}," resp.headers:\n",[178,1654,1655,1658,1661],{"class":88,"line":559},[178,1656,1657],{"class":227},"        etag_file.write_text(resp.headers[",[178,1659,1660],{"class":187},"\"ETag\"",[178,1662,1663],{"class":227},"])\n",[178,1665,1666,1668],{"class":88,"line":564},[178,1667,538],{"class":223},[178,1669,1670],{"class":227}," resp.content\n",[10,1672,1673,1674,1677],{},"A ",[175,1675,1676],{},"304 Not Modified"," costs a round trip instead of a download, which turns a slow hourly job into a fast one whenever the source has not changed.",[163,1679,1681],{"id":1680},"wrap-it-all-in-one-loader","Wrap it all in one loader",[10,1683,1684,1685,1688],{},"The pieces above belong together, because every caller wants the same behaviour: authenticate, retry, verify, parse, and fail with a message naming the source. One function does it, and the rest of the codebase never touches ",[175,1686,1687],{},"requests"," again:",[168,1690,1692],{"className":214,"code":1691,"language":216,"meta":173,"style":173},"\"\"\"loader.py — the only place the network meets the Excel reader.\"\"\"\nimport io\nimport os\n\nimport pandas as pd\nimport requests\nfrom requests.adapters import HTTPAdapter\nfrom urllib3.util.retry import Retry\n\nclass RemoteWorkbookError(RuntimeError):\n    pass\n\ndef _session() -> requests.Session:\n    session = requests.Session()\n    retry = Retry(total=4, backoff_factor=1.5,\n                  status_forcelist=(429, 500, 502, 503, 504),\n                  allowed_methods=frozenset([\"GET\"]))\n    session.mount(\"https:\u002F\u002F\", HTTPAdapter(max_retries=retry))\n    if token := os.getenv(\"REPORT_API_TOKEN\"):\n        session.headers[\"Authorization\"] = f\"Bearer {token}\"\n    return session\n\ndef load_remote_excel(url: str, **read_kwargs) -> pd.DataFrame:\n    try:\n        resp = _session().get(url, timeout=30)\n        resp.raise_for_status()\n    except requests.RequestException as exc:\n        raise RemoteWorkbookError(f\"{url}: {exc}\") from exc\n\n    if resp.content[:2] != b\"PK\":\n        preview = resp.content[:80].decode(\"utf-8\", \"replace\")\n        raise RemoteWorkbookError(f\"{url}: not a workbook — starts {preview!r}\")\n\n    read_kwargs.setdefault(\"engine\", \"openpyxl\")\n    return pd.read_excel(io.BytesIO(resp.content), **read_kwargs)\n",[175,1693,1694,1699,1705,1711,1715,1725,1731,1741,1751,1755,1770,1775,1779,1789,1798,1826,1855,1871,1886,1904,1929,1937,1942,1961,1969,1988,1994,2008,2044,2049,2069,2095,2128,2133,2148],{"__ignoreMap":173},[178,1695,1696],{"class":88,"line":180},[178,1697,1698],{"class":187},"\"\"\"loader.py — the only place the network meets the Excel reader.\"\"\"\n",[178,1700,1701,1703],{"class":88,"line":237},[178,1702,224],{"class":223},[178,1704,283],{"class":227},[178,1706,1707,1709],{"class":88,"line":244},[178,1708,224],{"class":223},[178,1710,627],{"class":227},[178,1712,1713],{"class":88,"line":290},[178,1714,241],{"emptyLinePlaceholder":240},[178,1716,1717,1719,1721,1723],{"class":88,"line":301},[178,1718,224],{"class":223},[178,1720,228],{"class":227},[178,1722,231],{"class":223},[178,1724,234],{"class":227},[178,1726,1727,1729],{"class":88,"line":308},[178,1728,224],{"class":223},[178,1730,197],{"class":227},[178,1732,1733,1735,1737,1739],{"class":88,"line":313},[178,1734,744],{"class":223},[178,1736,747],{"class":227},[178,1738,224],{"class":223},[178,1740,752],{"class":227},[178,1742,1743,1745,1747,1749],{"class":88,"line":339},[178,1744,744],{"class":223},[178,1746,759],{"class":227},[178,1748,224],{"class":223},[178,1750,764],{"class":227},[178,1752,1753],{"class":88,"line":380},[178,1754,241],{"emptyLinePlaceholder":240},[178,1756,1757,1760,1763,1765,1768],{"class":88,"line":386},[178,1758,1759],{"class":223},"class",[178,1761,1762],{"class":183}," RemoteWorkbookError",[178,1764,436],{"class":227},[178,1766,1767],{"class":326},"RuntimeError",[178,1769,1399],{"class":227},[178,1771,1772],{"class":88,"line":391},[178,1773,1774],{"class":223},"    pass\n",[178,1776,1777],{"class":88,"line":412},[178,1778,241],{"emptyLinePlaceholder":240},[178,1780,1781,1783,1786],{"class":88,"line":427},[178,1782,316],{"class":223},[178,1784,1785],{"class":319}," _session",[178,1787,1788],{"class":227},"() -> requests.Session:\n",[178,1790,1791,1794,1796],{"class":88,"line":459},[178,1792,1793],{"class":227},"    session ",[178,1795,250],{"class":223},[178,1797,651],{"class":227},[178,1799,1800,1803,1805,1808,1811,1813,1815,1817,1820,1822,1824],{"class":88,"line":488},[178,1801,1802],{"class":227},"    retry ",[178,1804,250],{"class":223},[178,1806,1807],{"class":227}," Retry(",[178,1809,1810],{"class":350},"total",[178,1812,250],{"class":223},[178,1814,788],{"class":326},[178,1816,330],{"class":227},[178,1818,1819],{"class":350},"backoff_factor",[178,1821,250],{"class":223},[178,1823,801],{"class":326},[178,1825,791],{"class":227},[178,1827,1828,1831,1833,1835,1837,1839,1841,1843,1845,1847,1849,1851,1853],{"class":88,"line":530},[178,1829,1830],{"class":350},"                  status_forcelist",[178,1832,250],{"class":223},[178,1834,436],{"class":227},[178,1836,820],{"class":326},[178,1838,330],{"class":227},[178,1840,825],{"class":326},[178,1842,330],{"class":227},[178,1844,830],{"class":326},[178,1846,330],{"class":227},[178,1848,835],{"class":326},[178,1850,330],{"class":227},[178,1852,840],{"class":326},[178,1854,843],{"class":227},[178,1856,1857,1860,1862,1864,1866,1868],{"class":88,"line":535},[178,1858,1859],{"class":350},"                  allowed_methods",[178,1861,250],{"class":223},[178,1863,853],{"class":326},[178,1865,856],{"class":227},[178,1867,859],{"class":187},[178,1869,1870],{"class":227},"]))\n",[178,1872,1873,1876,1878,1880,1882,1884],{"class":88,"line":559},[178,1874,1875],{"class":227},"    session.mount(",[178,1877,882],{"class":187},[178,1879,885],{"class":227},[178,1881,888],{"class":350},[178,1883,250],{"class":223},[178,1885,893],{"class":227},[178,1887,1888,1890,1893,1896,1899,1902],{"class":88,"line":564},[178,1889,415],{"class":223},[178,1891,1892],{"class":227}," token ",[178,1894,1895],{"class":223},":=",[178,1897,1898],{"class":227}," os.getenv(",[178,1900,1901],{"class":187},"\"REPORT_API_TOKEN\"",[178,1903,1399],{"class":227},[178,1905,1907,1910,1912,1914,1916,1918,1920,1922,1925,1927],{"class":88,"line":1906},20,[178,1908,1909],{"class":227},"        session.headers[",[178,1911,659],{"class":187},[178,1913,473],{"class":227},[178,1915,250],{"class":223},[178,1917,666],{"class":223},[178,1919,669],{"class":187},[178,1921,365],{"class":445},[178,1923,1924],{"class":227},"token",[178,1926,451],{"class":445},[178,1928,684],{"class":187},[178,1930,1932,1934],{"class":88,"line":1931},21,[178,1933,538],{"class":223},[178,1935,1936],{"class":227}," session\n",[178,1938,1940],{"class":88,"line":1939},22,[178,1941,241],{"emptyLinePlaceholder":240},[178,1943,1945,1947,1950,1952,1954,1956,1958],{"class":88,"line":1944},23,[178,1946,316],{"class":223},[178,1948,1949],{"class":319}," load_remote_excel",[178,1951,323],{"class":227},[178,1953,327],{"class":326},[178,1955,330],{"class":227},[178,1957,333],{"class":223},[178,1959,1960],{"class":227},"read_kwargs) -> pd.DataFrame:\n",[178,1962,1964,1967],{"class":88,"line":1963},24,[178,1965,1966],{"class":223},"    try",[178,1968,485],{"class":227},[178,1970,1972,1975,1977,1980,1982,1984,1986],{"class":88,"line":1971},25,[178,1973,1974],{"class":227},"        resp ",[178,1976,250],{"class":223},[178,1978,1979],{"class":227}," _session().get(url, ",[178,1981,351],{"class":350},[178,1983,250],{"class":223},[178,1985,54],{"class":326},[178,1987,259],{"class":227},[178,1989,1991],{"class":88,"line":1990},26,[178,1992,1993],{"class":227},"        resp.raise_for_status()\n",[178,1995,1997,2000,2003,2005],{"class":88,"line":1996},27,[178,1998,1999],{"class":223},"    except",[178,2001,2002],{"class":227}," requests.RequestException ",[178,2004,231],{"class":223},[178,2006,2007],{"class":227}," exc:\n",[178,2009,2011,2013,2016,2018,2020,2022,2024,2026,2028,2030,2033,2035,2037,2039,2041],{"class":88,"line":2010},28,[178,2012,430],{"class":223},[178,2014,2015],{"class":227}," RemoteWorkbookError(",[178,2017,439],{"class":223},[178,2019,442],{"class":187},[178,2021,365],{"class":445},[178,2023,448],{"class":227},[178,2025,451],{"class":445},[178,2027,371],{"class":187},[178,2029,365],{"class":445},[178,2031,2032],{"class":227},"exc",[178,2034,451],{"class":445},[178,2036,442],{"class":187},[178,2038,1326],{"class":227},[178,2040,744],{"class":223},[178,2042,2043],{"class":227}," exc\n",[178,2045,2047],{"class":88,"line":2046},29,[178,2048,241],{"emptyLinePlaceholder":240},[178,2050,2052,2054,2056,2058,2060,2063,2065,2067],{"class":88,"line":2051},30,[178,2053,415],{"class":223},[178,2055,467],{"class":227},[178,2057,470],{"class":326},[178,2059,473],{"class":227},[178,2061,2062],{"class":223},"!=",[178,2064,479],{"class":223},[178,2066,482],{"class":187},[178,2068,485],{"class":227},[178,2070,2072,2075,2077,2079,2082,2085,2088,2090,2093],{"class":88,"line":2071},31,[178,2073,2074],{"class":227},"        preview ",[178,2076,250],{"class":223},[178,2078,467],{"class":227},[178,2080,2081],{"class":326},"80",[178,2083,2084],{"class":227},"].decode(",[178,2086,2087],{"class":187},"\"utf-8\"",[178,2089,330],{"class":227},[178,2091,2092],{"class":187},"\"replace\"",[178,2094,259],{"class":227},[178,2096,2098,2100,2102,2104,2106,2108,2110,2112,2115,2117,2120,2122,2124,2126],{"class":88,"line":2097},32,[178,2099,430],{"class":223},[178,2101,2015],{"class":227},[178,2103,439],{"class":223},[178,2105,442],{"class":187},[178,2107,365],{"class":445},[178,2109,448],{"class":227},[178,2111,451],{"class":445},[178,2113,2114],{"class":187},": not a workbook — starts ",[178,2116,365],{"class":445},[178,2118,2119],{"class":227},"preview",[178,2121,520],{"class":223},[178,2123,451],{"class":445},[178,2125,442],{"class":187},[178,2127,259],{"class":227},[178,2129,2131],{"class":88,"line":2130},33,[178,2132,241],{"emptyLinePlaceholder":240},[178,2134,2136,2139,2142,2144,2146],{"class":88,"line":2135},34,[178,2137,2138],{"class":227},"    read_kwargs.setdefault(",[178,2140,2141],{"class":187},"\"engine\"",[178,2143,330],{"class":227},[178,2145,549],{"class":187},[178,2147,259],{"class":227},[178,2149,2151,2153,2155,2157],{"class":88,"line":2150},35,[178,2152,538],{"class":223},[178,2154,541],{"class":227},[178,2156,333],{"class":223},[178,2158,2159],{"class":227},"read_kwargs)\n",[10,2161,2162,2163,2166],{},"Decoding the first bytes into the error message is a small touch that pays for itself: the log line then reads ",[175,2164,2165],{},"not a workbook — starts '\u003C!DOCTYPE html>\u003Ctitle>Sign in'",", which diagnoses an expired token without anybody reproducing the run.",[20,2168,29,2173,29,2176,29,2179,29,2182,29,2187,29,2191,29,2197,29,2202,29,2205,29,2208,29,2211,29,2216,29,2218,29,2221],{"viewBox":2169,"role":23,"ariaLabelledBy":2170,"xmlns":27,"style":28},"0 0 760 198",[2171,2172],"url3-t","url3-d",[31,2174,2175],{"id":2171},"What the error message should contain",[35,2177,2178],{"id":2172},"A useful failure names the source, the status and the first bytes of the body, which together identify an expired token, a moved file or a wrong format without a rerun.",[39,2180],{"x":41,"y":41,"width":42,"height":2181,"fill":44},"198",[46,2183,2186],{"x":935,"y":2184,"style":2185},"28","font-size:12.5px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","unhelpful",[46,2188,2190],{"x":2189,"y":2184,"style":156},"566","actionable",[39,2192],{"x":54,"y":2193,"width":2194,"height":2193,"rx":2195,"fill":2196,"stroke":60},"46","332","9","#fee8f2",[46,2198,2201],{"x":935,"y":2199,"style":2200},"74","font-size:12px;fill:var(--accent-ink,#be185d);text-anchor:middle","BadZipFile: File is not a zip file",[39,2203],{"x":54,"y":2204,"width":2194,"height":2193,"rx":2195,"fill":2196,"stroke":60},"100",[46,2206,2207],{"x":935,"y":142,"style":2200},"no source, no status, no clue",[39,2209],{"x":2210,"y":2193,"width":2194,"height":2193,"rx":2195,"fill":152,"stroke":60},"398",[46,2212,2215],{"x":2213,"y":2199,"style":2214},"564","font-size:12px;fill:var(--teal-ink,#0b6157);text-anchor:middle","url + status + first bytes",[39,2217],{"x":2210,"y":2204,"width":2194,"height":2193,"rx":2195,"fill":152,"stroke":60},[46,2219,2220],{"x":2213,"y":142,"style":2214},"\"starts '\u003C!DOCTYPE html'\" — token expired",[46,2222,2224],{"x":48,"y":2223,"style":988},"178","The cost is one decode; the saving is a whole debugging session",[163,2226,2228],{"id":2227},"common-pitfalls-and-gotchas","Common pitfalls and gotchas",[2230,2231,2232,2240,2254,2263,2273],"ul",{},[2233,2234,2235,2239],"li",{},[2236,2237,2238],"strong",{},"Trusting a 200."," Portals return login pages with a 200 status. Check the bytes.",[2233,2241,2242,2245,2246,2249,2250,2253],{},[2236,2243,2244],{},"Streaming a zip."," ",[175,2247,2248],{},"get_object()[\"Body\"]"," and ",[175,2251,2252],{},"resp.raw"," are forward-only; buffer first.",[2233,2255,2256,2262],{},[2236,2257,2258,2259,18],{},"Forgetting ",[175,2260,2261],{},"seek(0)"," A buffer you just wrote is positioned at the end.",[2233,2264,2265,2268,2269,2272],{},[2236,2266,2267],{},"No timeout."," A ",[175,2270,2271],{},"requests.get"," without one can hang a scheduled job indefinitely.",[2233,2274,2275,2278],{},[2236,2276,2277],{},"Credentials in code."," Read them from the environment; a token in a repository outlives the job that needed it.",[163,2280,2282],{"id":2281},"performance-and-scale-notes","Performance and scale notes",[10,2284,2285,2286,2289,2290,18],{},"Network time usually dominates the parse, so the biggest wins are conditional requests and connection reuse rather than a faster reader — though ",[175,2287,2288],{},"engine=\"calamine\""," still helps on large files. When fetching many workbooks, run the downloads concurrently with a thread pool (they are I\u002FO-bound) and the parsing in processes (it is CPU-bound); mixing the two in one pool wastes both. That split is the same one described in ",[14,2291,2293],{"href":2292},"\u002Fadvanced-data-transformation-and-cleaning\u002Fworking-with-large-excel-files-in-python\u002Fprocess-multiple-excel-files-in-parallel-with-python\u002F","Process multiple Excel files in parallel with Python",[163,2295,2297],{"id":2296},"conclusion","Conclusion",[10,2299,2300,2301,2304],{},"Reading a workbook from a URL or a byte string is a one-liner surrounded by four guards worth having: a status check, a content-type check, a magic-byte check and a timeout. Buffer object-storage bodies rather than streaming them into the reader, stream genuinely large downloads to a temporary file, and use ",[175,2302,2303],{},"ETag"," caching when the same file is fetched on a schedule. Then the network becomes just another source, with no temporary files and no surprises.",[163,2306,2308],{"id":2307},"frequently-asked-questions","Frequently asked questions",[10,2310,2311,2314,2315,2318,2319,2321],{},[2236,2312,2313],{},"Can pandas read a URL directly?","\nYes, ",[175,2316,2317],{},"pd.read_excel(\"https:\u002F\u002F…\")"," works for a simple public file. Fetching with ",[175,2320,1687],{}," first is better in a job, because you can set headers and timeouts, check the status and content type, and retry — none of which the one-liner allows.",[10,2323,2324,2327,2328,2331,2332,2335],{},[2236,2325,2326],{},"Why does the read fail with \"format cannot be determined\"?","\nA ",[175,2329,2330],{},"BytesIO"," has no filename, so pandas cannot use the extension and falls back to sniffing the bytes. If the response was an HTML login page rather than a workbook, that sniff fails. Check the status code and content type first, and pass ",[175,2333,2334],{},"engine="," explicitly.",[10,2337,2338,2341,2342,2345],{},[2236,2339,2340],{},"Do I need to save the file to disk at all?","\nNo. Every reader accepts a file-like object, so ",[175,2343,2344],{},"io.BytesIO(response.content)"," is enough. Save to disk only when you want a cache or an audit copy.",[10,2347,2348,2351,2352,2354,2355,2358,2359,2361],{},[2236,2349,2350],{},"How do I read a workbook from S3?","\nDownload the object body into ",[175,2353,2330],{}," with ",[175,2356,2357],{},"download_fileobj"," and read that. Do not stream the body directly into the reader — an ",[175,2360,595],{}," needs random access to its zip directory.",[10,2363,2364,2367,2368,2371],{},[2236,2365,2366],{},"What about a very large download?","\nStream it to a temporary file with ",[175,2369,2370],{},"iter_content",", then read from the path. Holding a hundred megabytes in memory and then parsing it doubles the peak.",[163,2373,2375],{"id":2374},"related","Related",[2230,2377,2378,2386,2393,2400,2407],{},[2233,2379,2380,2245,2383,2385],{},[2236,2381,2382],{},"Up:",[14,2384,17],{"href":16}," — the formats these bytes may turn out to be.",[2233,2387,2388,2392],{},[14,2389,2391],{"href":2390},"\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-excel-file-format-cannot-be-determined-in-pandas\u002F","Fix \"Excel file format cannot be determined\" in pandas"," — the error a nameless buffer produces, and its fix.",[2233,2394,2395,2399],{},[14,2396,2398],{"href":2397},"\u002Fautomating-reporting-workflows\u002Fserving-excel-files-from-python-web-apps\u002Fbuild-an-excel-workbook-in-memory-with-bytesio\u002F","Build an Excel workbook in memory with BytesIO"," — the writing counterpart to everything here.",[2233,2401,2402,2406],{},[14,2403,2405],{"href":2404},"\u002Fadvanced-data-transformation-and-cleaning\u002Fmoving-data-between-excel-and-databases\u002Ffetch-api-data-into-excel-with-python-requests\u002F","Fetch API data into Excel with Python requests"," — fetching data rather than a whole workbook.",[2233,2408,2409,2413],{},[14,2410,2412],{"href":2411},"\u002Fautomating-reporting-workflows\u002Fpublishing-excel-reports-to-cloud-storage\u002Fupload-an-excel-report-to-amazon-s3-with-boto3\u002F","Upload an Excel report to Amazon S3 with boto3"," — putting a workbook back where this one reads it from.",[2415,2416,2417],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}html pre.shiki code .s-wDw, html code.shiki .s-wDw{--shiki-default:#6A737D;--shiki-dark:#BDC4CC}",{"title":173,"searchDepth":237,"depth":237,"links":2419},[2420,2421,2422,2423,2424,2425,2426,2427,2428,2429,2430,2431,2432,2433,2434],{"id":165,"depth":237,"text":166},{"id":207,"depth":237,"text":208},{"id":265,"depth":237,"text":266},{"id":611,"depth":237,"text":612},{"id":730,"depth":237,"text":731},{"id":992,"depth":237,"text":993},{"id":1113,"depth":237,"text":1114},{"id":1248,"depth":237,"text":1249},{"id":1474,"depth":237,"text":1475},{"id":1680,"depth":237,"text":1681},{"id":2227,"depth":237,"text":2228},{"id":2281,"depth":237,"text":2282},{"id":2296,"depth":237,"text":2297},{"id":2307,"depth":237,"text":2308},{"id":2374,"depth":237,"text":2375},"2026-08-27","Load a workbook straight from HTTP, S3, a database blob or an upload: BytesIO, content-type checks, authentication, retries, streaming downloads and caching.","md",[2439,2441,2443,2445,2447],{"q":2313,"a":2440},"Yes, pd.read_excel(\"https:\u002F\u002F…\") works for a simple public file. Fetching with requests first is better in a job, because you can set headers and timeouts, check the status and content type, and retry — none of which the one-liner allows.",{"q":2326,"a":2442},"A BytesIO has no filename, so pandas cannot use the extension and falls back to sniffing the bytes. If the response was an HTML login page rather than a workbook, that sniff fails. Check the status code and content type first, and pass engine= explicitly.",{"q":2340,"a":2444},"No. Every reader accepts a file-like object, so io.BytesIO(response.content) is enough. Save to disk only when you want a cache or an audit copy.",{"q":2350,"a":2446},"Download the object body into BytesIO with get_object or download_fileobj and read that. Do not stream the body directly into the reader — an xlsx needs random access to its zip directory.",{"q":2366,"a":2448},"Stream it to a temporary file with iter_content, then read from the path. Holding a hundred megabytes in memory and then parsing it doubles the peak.",{},"\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-an-excel-file-from-a-url-or-bytes-in-python",{"title":2452,"description":2453},"Read Excel from a URL or Bytes","Read xlsx without saving to disk: requests plus BytesIO, verifying the response is really a workbook, auth headers, retries, S3 objects, uploads and local caching.","read-an-excel-file-from-a-url-or-bytes-in-python","getting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-an-excel-file-from-a-url-or-bytes-in-python\u002Findex","how-to","EGfiqQLdPcUFb8rA4JZE4hc8QfSOatWZwinA3RJJLWg",[2459,2463],{"title":2460,"path":2461,"stem":2462,"children":-1},"Convert .xls to .xlsx with Python","\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fconvert-xls-to-xlsx-with-python","getting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fconvert-xls-to-xlsx-with-python\u002Findex",{"title":2464,"path":2465,"stem":2466,"children":-1},"Read and Write .ods Files with Python","\u002Fgetting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-and-write-ods-files-with-python","getting-started-with-python-excel-automation\u002Fhandling-excel-file-formats-and-conversions\u002Fread-and-write-ods-files-with-python\u002Findex",1788710159619]