The basic workflow
Open DevTools (your browser's built-in developer panel, usually F12) and switch to the Network tab, then filter to Fetch/XHR — these are the background data requests the page makes. Now do the action you want to scrape: load a page, scroll, run a search. Scan the requests for ones that return structured JSON containing the data you want. Right-click that request and choose "Copy as cURL" (cURL is a command-line tool for making HTTP requests) — you now have a known-good copy. Paste it into a script, confirm it works, then remove headers one by one to find the minimum set the server actually needs.
Handling auth and CSRF
Most internal APIs want proof of who you are: usually a session cookie (a token tying requests to your login), a CSRF token from the initial page (a one-time value that proves the request came from the real site, not a forgery), or an auth header. Session cookies: load the public page first, grab the cookie, reuse it. CSRF tokens: pull the token out of the initial HTML (usually a meta tag or a hidden form input) and include it in later API calls. Bearer tokens: log in once through the normal flow, capture the token, and refresh it as needed.
When reverse-engineering fails
Some endpoints fight back. They might sign each request with an HMAC (a tamper-proof checksum) computed in deliberately scrambled, or obfuscated, JavaScript; attach device-attestation tokens that only exist if you actually run the page's JS; or only serve the mobile app, locked down with TLS pinning (where the app refuses any https connection it does not specifically trust). In those cases the effort of reverse-engineering outweighs just rendering the page in a real browser — so fall back to that. Mobile API endpoints are their own category and usually need MITM proxy work — sitting between the app and the server to inspect traffic — using a tool like Mitmproxy or Charles on a real device.