Drive the real website with Playwright — filter to a row, tick its checkbox, click Download, and let the browser save the file.

The download fires from a JavaScript event, not a plain href. expect_download() resolves only once the browser commits the transfer.

save_as() blocks on the entire multi-gigabyte transfer, one file at a time. With ~38 files at 10–40 GB each, a serial save loop would run for days and die on the first dropped connection. Decouple finding the file from moving its bytes.

Let the browser do only what it is good at — minting signed links — then hand those links to a purpose-built downloader.

Every real download request hits a URL containing 'download' or 'dd/'. Listening on page requests captures the signed link the moment the UI generates it.

aria2c moves bytes far better than a headless browser: 16 parallel connections, segmented ranges, and -c to resume a half-finished file. The architecture splits cleanly — browser finds, aria2c fetches — but the signed links still expire fast, so the two halves must run close together.

Skip the browser entirely and use Egnyte's official Python SDK to list and download the folder.

Every method on EgnyteClient expects an access_token. A public, anonymous share link has no token and no account behind it.

Native paths are built for authenticated tenants, not anonymous public links. There is no credential to supply, so the SDK can never authenticate. The way in is the same unauthenticated API the public web UI itself calls.

Open F12, watch the network tab while the page loads, and hit the REST endpoint the UI is quietly calling.

The page renders from a JSON payload. The visible UI is just a skin over a REST API that returns the folder structure as data.

First direct contact with the backend. Talking to the API instead of the rendered page removes the browser, the virtualized DOM, and the per-click overhead — but the API has its own rules about paths, methods, and headers that have to be learned one error at a time.

Walk the tree breadth-first: POST a path, read its children from contents.results, queue folders, record files.

An empty or "/" path returns 403 FOLDER_ACCESS_FORBIDDEN — enumeration has to start at the named root, not the share root. A persistent Session and a polite delay survive the rate limiter.

All 38 files get mapped successfully — but the listing payload carries no download URL. Knowing a file exists is not the same as being able to fetch it. A separate call must resolve each path into a downloadable link.

Probe candidate endpoints to turn a known file path into a real download URL.

/download and /download_info return 404 — they do not exist. fsi-content returns 500, not 404: the endpoint is real, but it is being called the wrong way.

A 500 is a clue, not a wall. A 404 means 'wrong door'; a 500 means 'right door, wrong key.' fsi-content is the resolver — it just needs something the bare request is not sending yet.

Copy the browser's headers — Referer, Origin, User-Agent, X-Requested-With — so the request looks like it came from the page.

Still 500. Cosmetic headers are not what the server checks. It wants a per-session XSRF token that the page's JavaScript mints at runtime, plus the matching session cookie.

Looking like the browser is not the same as holding the browser's session state. The token cannot be guessed or hardcoded — it only exists after a real browser has loaded the page. The browser has to be kept in the loop just long enough to mint it.

Let the browser start each download to mint the signed link, capture the URL, then immediately cancel the stream before any bytes move.

page.on('download') hands over the signed URL at the instant the transfer begins; download.cancel() throws the bytes away. The link is the prize, not the file.

Defeated by the UI itself: the virtualized DOM renders no stable rows or buttons to click, there is no right-click 'Copy Link Address', the nested SRL-2018 folder is missed entirely, and selecting two or more files triggers a batch-download error. Clicking through the UI does not scale — the API handshake has to be reproduced directly.

Right-click the real request in DevTools → "Copy as cURL" and read the exact handshake the page performs.

The cURL export reveals three chained calls and the two secrets that make them work: an x-egnyte-xsrftoken header and a JSESSIONID cookie. fsi-content returns each item's real link as _self.linkedResources.itemDownload.url.

This is the breakthrough. The full sequence is now visible end to end — LIST a folder, RESOLVE each item into a /dd/?entryId= link, then DOWNLOAD with cookie auth following the redirect to the CDN. Everything after this is just automating what the browser was already doing.

Turn every dead end into structured intel — phase, blocker, the DevTools clue that exposed it, and why it happens.

Each failure mode maps to a specific server behavior: hanging events, 500s, connection caps, expiring tokens, and the session handshake.

Writing the blockers down as data, not prose, makes the final design fall out almost mechanically — every row points directly at a countermeasure. The full table is rendered in Section B below.

Distill every lesson into a single canonical header set, with the XSRF token minted live rather than guessed.

The minimal working request needs exactly: a real User-Agent, the live x-egnyte-xsrftoken, the JSON content type, and the matching Origin/Referer for this specific link.

The token is no longer a constant to hardcode — it is captured from a live browser session and injected into every API call. This is the bridge between 'the browser can do it' and 'a script can do it.'

A three-phase hybrid: a real browser bootstraps the session once, then plain HTTP does all the heavy lifting.

Phase 1 mints JSESSIONID + x-egnyte-xsrftoken via Playwright. Phase 2 paginates the listing over httpx. Phase 3 streams each entryId serially with 1 MB chunks and resume support.

The browser is used for exactly one thing — minting credentials — and then gets out of the way. Everything heavy runs over raw HTTP: paginated, serial, resumable. 225 GB harvested without tripping the bot defenses that killed every earlier attempt.