From a9df56add060b6dab4f20ba1f6af42cf540c4bb4 Mon Sep 17 00:00:00 2001 From: Jan Tuomi Date: Thu, 22 May 2025 12:20:41 +0300 Subject: Update README --- README.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 75cff89..7560e84 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,8 @@ Ever wanted a zero-dependency, filesystem-backed, lock-free, durable, concurrent, retrying task queue implementation? No? +Note: This is pre-release software. Backwards compatibility will be guaranteed after v1.0. + ## Example ```python @@ -36,18 +38,23 @@ Or just chuck the `femtoqueue.py` file into your Python 3 project. There are no ## Features -This mini-library provides the `FemtoQueue` class with the standard queue interface: +This mini-library provides the `FemtoQueue` class with the standard queue interface, with some additions: -| Method | Description | -| :------------------------- | :---------------------------------- | -| `push(task: bytes) -> str` | Add a task to the queue, returns id | -| `pop() -> FemtoTask` | Get a task from the queue | +| Method | Description | +| :------------------------------------------- | :-------------------------------------- | +| `push(task: bytes) -> str` | Add a task to the queue, returns id | +| `pop() -> FemtoTask` | Get a task from the queue | +| `schedule(task: bytes, time_us: int) -> str` | Schedule a task for the given timestamp | +| `done(task: FemtoTask)` | Mark a task as done | +| `fail(task: FemtoTask)` | Mark a task as failed | Each task corresponds to one file in the `data_dir` directory. State changes are atomic since they use `rename()`. The task can contain whatever you want, the queue does not inspect it in any way. Each concurrent worker node (library user) must have a stable identifier `node_id`. This way workers can automatically retry a task if they unexpectedly crash in the middle of processing. -Stale tasks (i.e. in progress for too long) are moved back to `pending` automatically after a timeout is reached (default: 30s). +Scheduled tasks are moved to the back of the `pending` queue when the specified wall-clock time is less than the current wall-clock time. Stale tasks (i.e. in progress for too long) are moved back to `pending` automatically after a timeout is reached (default: 30s). These events are processed during a `pop()` call; the library does not run any background jobs by itself. + +Tasks are ordered using the system-provided monotonic clock to avoid issues such as NTP-related clock skew or daylight savings time moving the clock backwards in time, which would be possible if ordering was based on wall-clock time. As long as all worker nodes have the same clock source, pending tasks are guaranteed to be processed in insertion order, with microsecond precision. ## But isn't this slow? -- cgit v1.3