The Apache Pulsar community releases version 2.9.2! 60 contributors provided improvements and bug fixes that delivered 317 commits.
Highlights of this release are as below:
- Transactions performance test tool is available. PR-11933
- Brokers decrease the number of unacked messages. PR-13383
- Readers continue to read data from the compacted ledgers. PR-13629
This blog walks through the most noteworthy changes grouped by the affected functionalities. For the complete list including all features, enhancements, and bug fixes, check out the Pulsar 2.9.2 Release Notes.
Notable bug fixes and enhancements
Previously, when topics were unloaded, some data was lost to be read by readers if they have consumed some messages from some compacted ledgers.
Rewound the reader cursor to the next message of the mark delete position if
readCompacted = true.
Previously, brokers did not decrease the number of unacked messages if batch ack was enabled. Consequently, consumers were blocked if they reached
Decreased the number of unacked messages when
individualAckNormal was called.
Previously, chunked messages could not be queried through Pulsar SQL.
Add a chunked message map in
PulsarRecordCursor to maintain incomplete chunked messages. If one chunked message was received completely, it would be offered in the message queue to wait for deserialization.
Previously, Pulsar didn't support enabling or disabling schema upload at the broker level.
Added the configuration
isSchemaAutoUploadEnabled on the broker side.
Previously, readers were not able to read the latest messages in compacted topics if readers enabled
readCompacted and all the data of topics has been compacted to compacted ledgers.
forceReset configuration for the managed cursor, so that the cursor could be reset to a given position and readers can read data from compacted ledgers.
Previously, the wrong transaction sequenceId was recovered due to incorrect
ManagedLedgerInterceptor to update current sequenceId to
managedLedger properties and more.
Previously, it was hard to test transaction performance (such as the delay and rate of sending and consuming messages) when opening a transaction.
PerformanceTransaction class to support this enhancement.
Previously, Pulsar proxy would get into a state where it stopped proxying broker connections while Admin API proxying kept working.
Optimized the proxy connection to fail-fast if the target broker was not active, added connect timeout handling to proxy connection, and more.
After the method
getPendingQueueSize() was called and the send receipt came back, the peek from the
pendingMessages might get NPE during the process.
Added a thread-safe message count object in
OpSendMsgQueue for each compute process.
Previously, if a class was dynamically loaded by
ClassNotFoundException occurred when it was used by the default class load.
Changed context class loader through
Thread.currentThread().setContextClassLoader(classLoader) before every plugin calling back and changed the context class loader back to original class loader afterwards.
If you are interested in learning more about Pulsar 2.9.2, you can download and try it out now!
Pulsar Summit San Francisco 2022 will take place on August 18th, 2022. Register now and help us make it an even bigger success by spreading the word on social media!