◆ addMonitoringTag()
| virtual void facebook::logdevice::AsyncReader::addMonitoringTag |
( |
std::string |
tag | ) |
|
|
pure virtual |
Adds a monitoring tag to this reader. Monitoring tags are used to help identify readers that belong to a particular user, use case, function, etc for monitoring purposes.
Only affects subsequent startReading() calls.
◆ doNotDecodeBufferedWrites()
| virtual void facebook::logdevice::AsyncReader::doNotDecodeBufferedWrites |
( |
| ) |
|
|
pure virtual |
◆ doNotSkipPartiallyTrimmedSections()
| virtual void facebook::logdevice::AsyncReader::doNotSkipPartiallyTrimmedSections |
( |
| ) |
|
|
pure virtual |
If called, when reading a section of the log that has been partially trimmed, the reader will deliver whatever records are still available, which (because of LogDevice's distributed and nondeterministic nature) results in an interleaved stream of records and TRIM gaps, which is undesirable in some cases.
The default behaviour is to deliver a large trim gap for the entire section.
See doc/partially-trimmed.md for a detailed explanation.
◆ forceNoSingleCopyDelivery()
| virtual void facebook::logdevice::AsyncReader::forceNoSingleCopyDelivery |
( |
| ) |
|
|
pure virtual |
If called, disable the single copy delivery optimization even if the log is configured to support it. Each data record will be sent by all storage nodes that have a copy instead of exactly one. This greatly increases read availability at the cost of higher network bandwith and cpu usage.
Only affects subsequent startReading() calls.
◆ getBytesBuffered()
| virtual void facebook::logdevice::AsyncReader::getBytesBuffered |
( |
std::function< void(size_t)> |
callback | ) |
|
|
pure virtual |
Report the size (in bytes) of the data records that the underlying ClientReadStreamBuffers currently occupy
◆ includeByteOffset()
| virtual void facebook::logdevice::AsyncReader::includeByteOffset |
( |
| ) |
|
|
pure virtual |
If called, data records read by this AsyncReader will start including approximate amount of data written to given log up to current record once it become available to AsyncReader.
The value itself stored in DataRecord::attrs::byte_offset. Set as BYTE_OFFSET_INVALID if unavailable to AsyncReader yet.
Only affects subsequent startReading() calls.
◆ isConnectionHealthy()
| virtual int facebook::logdevice::AsyncReader::isConnectionHealthy |
( |
logid_t |
| ) |
const |
|
pure virtual |
Checks if the connection to the LogDevice cluster for a log appears healthy. When a read() call times out, this can be used to make an informed guess whether this is because there is no data or because there a service interruption.
NOTE: this is not 100% accurate but will expose common issues like losing network connectivity.
- Returns
- On success, returns 1 if the connection appears healthy or 0 if there are issues talking to the cluster. On error returns -1 and sets err to NOTFOUND (not reading given log).
◆ nextFromLsnWhenStuck()
| static lsn_t facebook::logdevice::AsyncReader::nextFromLsnWhenStuck |
( |
lsn_t |
stuck_lsn = LSN_INVALID, |
|
|
lsn_t |
tail_lsn = LSN_INVALID |
|
) |
| |
|
static |
Get next lowest recommended LSN to read from when servers appear stuck.
To force progress when stuck at stuck_lsn, call stopReading(), then call startReading() from nextFromLsnWhenStuck(stuck_lsn).
NOTE: Forcing progress is likely to miss some records between stuck_lsn and nextFromLsnWhenStuck(stuck_lsn). The records will become available eventually (once server-side issues have been resolved), but the only way to read them is to re-read the section from stuck_lsn to nextFromLsnWhenStuck(stuck_lsn) later.
- Parameters
-
| stuck_lsn | LSN at which the reader got stuck; that is, the last LSN the reader tried to read or managed to read (+-1 does not affect the output of this function). |
| tail_lsn | Tail LSN, if known, as returned by Client::getTailLSN() or Client::findTime(). |
- Returns
- Recommended LSN to use as
from_lsn in startRead() to make progress, no less than stuck_lsn.
◆ resumeReading()
| virtual int facebook::logdevice::AsyncReader::resumeReading |
( |
logid_t |
log_id | ) |
|
|
pure virtual |
Requests delivery for a log to resume after a previous delivery was declined (callback returned false). This can be used to avoid waiting on the redelivery timer when the callback becomes ready to accept new records.
NOTE: involves interthread communication which can fail if the queues fill up. However, no failure handling is generally needed because delivery is retried on a timer.
- Parameters
-
| log_id | log ID to stop reading |
- Returns
- 0 is returned if resume request was successfully enqueued for delivery. On failure -1 is returned and logdevice::err is set to NOBUFS if request could not be enqueued because a buffer space limit was reached NOTFOUND if reading was not started for specified log
◆ setDoneCallback()
| virtual void facebook::logdevice::AsyncReader::setDoneCallback |
( |
std::function< void(logid_t)> |
| ) |
|
|
pure virtual |
Sets a callback that the LogDevice client library will call when it has finished reading the requested range of LSNs.
◆ setGapCallback()
| virtual void facebook::logdevice::AsyncReader::setGapCallback |
( |
std::function< bool(const GapRecord &)> |
| ) |
|
|
pure virtual |
Sets a callback that the LogDevice client library will call when a gap record is delivered for this log. A gap record informs the reader about gaps in the sequence of record numbers. In most cases such gaps are benign and not an indication of data loss. See class GapRecord in Record.h for details.
The callback should return true if the gap was successfully consumed. If the callback returns false, delivery of the same gap will be retried after some time. Redelivery can also be requested with a resumeReading() call.
◆ setHealthChangeCallback()
| virtual void facebook::logdevice::AsyncReader::setHealthChangeCallback |
( |
std::function< void(logid_t, HealthChangeType)> |
| ) |
|
|
pure virtual |
Sets a callback that the LogDevice client library will call when a health change is detected for any log it is reading from.
The callback receives as parameters the log ID for which the change was detected and a status reporting the type of health change.
◆ setRecordCallback()
| virtual void facebook::logdevice::AsyncReader::setRecordCallback |
( |
std::function< bool(std::unique_ptr< DataRecord > &)> |
| ) |
|
|
pure virtual |
Sets a callback that the LogDevice client library will call when a record is read.
The callback should return true if the record was successfully consumed. If the callback returns false, delivery of the same record will be retried after some time. Redelivery can also be requested with a resumeReading() call.
NOTE: The callback must not drain the input unique_ptr& if it return false (this is asserted in debug builds).
Only affects subsequent startReading() calls; calling startReading() first and setRecordCallback() after has no effect.
◆ startReading()
| virtual int facebook::logdevice::AsyncReader::startReading |
( |
logid_t |
log_id, |
|
|
lsn_t |
from, |
|
|
lsn_t |
until = LSN_MAX, |
|
|
const ReadStreamAttributes * |
attrs = nullptr |
|
) |
| |
|
pure virtual |
Start reading records from a log in a specified range of LSNs. The function will return as soon as the request is put on a local queue. Upon successful return, the next record to be delivered to a callback will be as described in
- Parameters
-
If the log is already being read by this AsyncReader, this method stops reading and starts again with the new parameters. However, the stopping is asynchronous. For a short time you may see records from both the old and the new stream, interleaved arbitrarily. If you need a clean cutover, call stopReading() (passing a callback), then startReading() after the callback is called.
- Parameters
-
| log_id | log ID to start reading |
| from | log sequence number (LSN) to move the read pointer to. If this LSN identifies a data record currently in the log, that record will be the next one delivered to a data callback installed for the log, or to a Reader object for the log. |
If the lowest (oldest) LSN in the log is greater than this value, the read pointer will move to the oldest record in the log and that record will be the next one delivered. See LSN_OLDEST in types.h.
If from falls into a gap in the numbering sequence, the next record delivered to this reader will be the gap record.
- Parameters
-
| until | the highest LSN the LogDevice cluster will deliver to this AsyncReader object. Once this LSN is reached, the LogDevice client library will call the done callback. The client must call startReading() again in order to continue delivery. If the read pointer comes across a sequence gap that includes this LSN, the delivery stops after the gap record is delivered. By default (see LSN_MAX in types.h) records continue to be delivered in sequence until delivery is explicitly cancelled by a call to stopReading() below, or altered by another call to startReading(). |
| attrs | Structure containing parameters that alter the behavior of the read stream. In particular it is used to pass filters for the server-side filtering experimental feature. |
- Returns
- 0 is returned if the request was successfully enqueued for delivery. On failure -1 is returned and logdevice::err is set to NOBUFS if request could not be enqueued because a buffer space limit was reached INVALID_PARAM if from > until or the record callback was not specified. SHUTDOWN the logdevice::Client instance was destroyed. INTERNAL An internal error has been detected, check logs.
◆ stopReading()
| virtual int facebook::logdevice::AsyncReader::stopReading |
( |
logid_t |
log_id, |
|
|
std::function< void()> |
callback = nullptr |
|
) |
| |
|
pure virtual |
Ask LogDevice cluster to stop delivery of this log's records. The callbacks registered for the log or the Reader object reading records from this log will stop receiving this log's records until one of startReading() methods is called again.
The function returns as soon as the request is put on a local queue. However, record/gap callbacks may continue to be called for the log until a Client thread is able to process the stop request. After the optional callback is called (on the Client thread), it is guaranted that no further records or gaps will be delivered. The callback is called exactly once, but might be called during or after AsyncReader destruction.
- Parameters
-
| log_id | log ID to stop reading |
| callback | optional callback to invoke when the request has taken effect and no more records will be delivered |
- Returns
- 0 is returned if a stop request was successfully enqueued for delivery. On failure -1 is returned and logdevice::err is set to NOTFOUND if reading was not started for specified log
◆ withoutPayload()
| virtual void facebook::logdevice::AsyncReader::withoutPayload |
( |
| ) |
|
|
pure virtual |
If called, data records read by this AsyncReader will not include payloads.
This makes reading more efficient when payloads are not needed (they won't be transmitted over the network).
Only affects subsequent startReading() calls.
The documentation for this class was generated from the following file: