|
LogDevice API
|
Utility class for buffering and batching appends on the client. More...
#include <chrono>#include <memory>#include <tuple>#include <variant>#include <vector>#include "logdevice/include/Client.h"#include "logdevice/include/types.h"Go to the source code of this file.
Namespaces | |
Utility class for buffering and batching appends on the client.
The regular Client::append() immediately sends the record to LogDevice. Because of the per-append cost of processing inside LogDevice, sending many small records can limit throughput.
This class allows latency of writes to be traded off for throughput. It presents a similar API to Client::append() but buffers appends for the same log on the client and sends them to LogDevice in fewer, larger, records. The records are automatically decoded on the read path by Reader.
BufferedWriter appends are by necessity async so there is a callback interface to notify the application when an append has completed. Because BufferedWriter is meant for high-throughput writing, the callback interface does not use std::function but a slightly more complicated setup: the application provides a single subclass of AppendCallback when it creates BufferedWriter.
When it calls BufferedWriter::append(), the application may, optionally, provide a pointer to a piece of context. This pointer is included, along with the payload, in the ContextSet vector at callback.
Applications are expected to configure the latency tradeoff via Options::time_trigger. For example, a value of 1 second means that buffered writes for a log will be flushed when the oldest of them has been buffered for 1 second. With a steady stream of appends to the log, we will essentially flush once every second.
See Options for additional features:
All methods in this class are thread-safe.
See doc/buffered-writer.md for an overview of the implementation.