Module httpasync
Non-blocking HTTP(S) downloads and a coroutine-based concurrent scheduler.
Unlike httpclient.lua (Turbo's I/O loop), this works without Turbo: it
drives LuaSocket/LuaSec sockets in non-blocking mode, yielding to a
socket.select-based scheduler so one Lua state can interleave many
downloads.
Sources: - The coroutine scheduler is based on https://www.lua.org/pil/9.4.html - The HTTP request/response handling (request lines, header parsing, redirects, chunked and content-length body reading) is adapted from LuaSocket (http.lua / the LTN12/http client).
Functions
| HttpAsync.fetch_url (url, redirect_count, headers) | Fetch one URL over a raw non-blocking socket, yielding while it would block so the caller can interleave many downloads. |
| HttpAsync.fetch_many (tasks, opts) | Download many tasks concurrently via a coroutine scheduler. |
Functions
- HttpAsync.fetch_url (url, redirect_count, headers)
-
Fetch one URL over a raw non-blocking socket, yielding while it would
block so the caller can interleave many downloads.
Parameters:
- url string
- redirect_count number internal recursion guard
- headers table optional extra request headers (e.g. cookie)
Returns:
-
(true, content, content_type) or (false, err)
- HttpAsync.fetch_many (tasks, opts)
-
Download many tasks concurrently via a coroutine scheduler.
Each task is a caller-supplied value;
opts.get_url(task)yields its URL. The scheduler runs up toopts.concurrencyfetches at once, resuming each coroutine wheneversocket.selectsays its socket is ready.Yield contract: a fetch coroutine either yields a socket object together with a "r"/"w" mode (meaning "I'm blocked waiting on this socket, resume me when it's ready"), or yields the sentinel
DONEfollowed by (item, success, content) to signal completion. The scheduler never sees the downloaded content as a yield value, so a response body cannot be mistaken for the completion sentinel.Parameters:
- tasks list of arbitrary task values
- opts table: concurrency number max simultaneous fetches (default 10) get_url(task) -> string (default: identity, i.e. tasks are URL strings) fetch(url) -> (success, content) (default: HttpAsync.fetchurl) onsuccess(task, content) called for each successful download onfailure(task, err) called for each failed download onprogress(completed, total) -> bool called periodically; return false to cancel
Returns:
-
true if all tasks completed, false if cancelled