Skip to main content

Apache Pulsar 2.10.2

2022-10-24​

Important notice​

  • [fix][admin] Fix get non-persistent topics issue in Namespaces. #16170

Broker​

  • [fix][broker] ManagedLedger: move to FENCED state in case of BadVersionException #17736
  • [fix][broker] Issue 17588: Allow deletion of a namespace that was left in deleted status #17592
  • [fix][broker] Fix BookKeeper packages npe #17291
  • [fix][broker] Fix rack awareness cache expiration data race #16825
  • [fix][broker] Fix stats-internal with option -m cause active ledger recover then close #16662
  • [fix][broker] Fix deadlock in key-shared dispatcher #16660
  • [fix][broker] PulsarLedgerManager: add missed return statement #16607
  • [fix][broker] Ensure the ack-timeout task gets re-scheduled when there is an exception in the final stage #16337
  • [fix][broker] Fix select broker is none #16316
  • [fix][broker] Fixed deadlock when checking topic ownership #16310
  • [fix][broker] Fix thread safety issues in accessing ManagedCursorContainer.heap ArrayList #16049
  • [fix][broker] Avoid IllegalStateException while client_version is not set #16788
  • [fix][broker] BadVersionException when splitting bundles, delay 100ms and try again. #16612
  • [fix][broker] Expose timestamp field for SchemaData&SchemaInfo #16380
  • [fix][broker] Fix NPE when drop backlog for time limit. #16235
  • [fix][broker] Fix NPE when get /admin/v2/namespaces/public/default/maxTopicsPerNamespace #16076
  • [fix][broker] Fix RawReader out of order #16390
  • [fix][broker] Fix calculate avg message per entry #17046
  • [fix][broker] Fix compaction subscription acknowledge Marker msg issue. #16205
  • [fix][broker] Fix consumer does not abide by the max unacks limitation for Key_Shared subscription #16718
  • [fix][broker] Fix consumer does not abide by the max unacks limitation for Shared subscription #16670
  • [fix][broker] Fix create client with TLS config #16014
  • [fix][broker] Fix namespace backlog quota check with retention. #17706
  • [fix][broker] Fix out of order data replication #17154
  • [fix][broker] Fix passing incorrect authentication data #16201
  • [fix][broker] Fix potential exception causing the reader to be unable to continue reading #17556
  • [fix][broker] Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath. #17192
  • [fix][broker] Fix schema does not replicate successfully #17049
  • [fix][broker] Fix the replicator unnecessary get schema request for BYTES schema #17523
  • [fix][broker] Fixed error when delayed messages trackers state grows to >1.5GB #16490
  • [fix][broker] Increment topic stats outbound message counters and update rate after messages have been written to the TCP/IP connection #17043
  • [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
  • [fix][broker] Multiple consumer dispatcher stuck when unackedMessages greater than maxUnackedMessages #17483
  • [fix][broker] PulsarLedgerManager to pass correct error code to BK client #16857
  • [fix][broker] Release the entry in getEarliestMessagePublishTimeOfPos. #16386
  • [fix][broker] Retry to delete the namespace if new topics are created during the namespace deletion #16676
  • [fix][broker] Retry when DistributedIdGenerator has BadVersion error #16491
  • [fix][broker] Skip reading more entries for a pending read with no more entries #16400
  • [fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic configuration #16408
  • [fix][broker] The configuration loadBalancerNamespaceMaximumBundles is invalid #16552
  • [fix][broker] Topic policy reader can't recover when getting any exception. #17562
  • [fix][broker] Unregister topic policy listener if managed ledger close failed #17652
  • [fix][broker] Upgrade log4j2 version to 2.18.0 #16884
  • [fix][broker] fix No such ledger exception #16420
  • [fix][broker] fix broker unackmessages become a negative number #17003
  • [fix][broker] remove exception log when access status.html #17025
  • [fix][broker] Consumer can't consume messages because there are two same topics in one broker #17526
  • [fix][broker] Fix getInternalStats occasional lack of LeaderInfo again #16238
  • [fix][broker] Fix subscribe dispatcher limiter not be initialized #16175
  • [fix][broker] Fix topic policies update not check message expiry #15941
  • [fix][broker] Fix topic-level replicator rate limiter not init #15825
  • [fix][broker] ManagedLedger metrics fail cause of zero period #17257
  • [fix][broker] Prevent StackOverFlowException in SHARED subscription #16968
  • [fix][broker] Fix message ack rate #16108
  • [fix][broker] Fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true #16219
  • [fix][broker] Fix npe when invoking replaceBookie. #16239
  • [fix][broker] Fail to update partition meta of topic due to ConflictException: subscription already exists for topic #17488
  • [fix][broker] Fix NPE when ledger id not found in OpReadEntry #15837
  • [Fix][broker] Fix topic dispatch rate limiter not init on broker-level #16084
  • [opt][broker] Optimize topic policy with HierarchyTopicPolicies about replicatorDispatchRate #14161
  • [fix][broker] Fix the broker close hanged issue. #17689
  • [fix][broker] Fix topic dispatch rate limiter not init on broker-level #16084 #17000
  • [fix][broker] Prevent StackOverFlowException in SHARED subscription #17053
  • [fix][broker] skip mis-configured resource usage(>100%) in load balancer #16937
  • [fix][broker] Fix wrong unit of NIC speed on Linux #15304
  • [fix][broker] Fix RawReader hasMessageAvailable returns true when no messages #16443
  • [improve][broker] Expose topic level storage write and read rate metrics #16855
  • [improve][broker] Broker/EntryFilter: make the delay for RESCHEDULED messages configurable (dispatcherFilterRescheduledMessageDelay) #16602
  • [improve][broker] Use OrderedExecutor instead of OrderedScheduler for consumer dispatch #16115
  • [improve][broker] Avoid going through all the consumers to get the message ack owner #16245
  • [improve][broker] Avoid reconnection when a partitioned topic was created concurrently #16043
  • [improve][broker] Cancel the loadShedding task when closing pulsar service #17632
  • [improve][broker] Consolidate Netty channel flushes to mitigate syscall overhead #16361
  • [improve][broker] Improve cursor.getNumberOfEntries if isUnackedRangesOpenCacheSetEnabled=true #17465
  • [improve][broker] Make MessageRedeliveryController work more efficiently #17804
  • [improve][broker] Make PulsarWebResource#getOwnerFromPeerClusterList async. #15940
  • [improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) #16286
  • [improve][broker] Recycle OpReadEntry in some corner cases #16399
  • [improve][broker] Reduce the consumers list sort by priority level #16243
  • [improve][broker] Reduce the re-schedule message read operation for PersistentDispatcherMultipleConsumers #16241
  • [improve][broker] Use LinkedHashSet for config items of type Set to preserve elements order #16138
  • [improve][broker] Use shared broker client scheduled executor provider #16338
  • [improve][broker] Remove unnecessary lock on the stats thread #16983
  • [improve][broker] Tidy update subscriptions dispatcher rate-limiter #16778
  • [improve][broker] PIP-149: Make resetCursor async (#16355) #16774
  • [improve][broker] added loadBalancerReportUpdateMinIntervalMillis and ignores memory usage in getMaxResourceUsage() #17598
  • [cleanup][broker] Follow up on #16968 to restore some behavior in PersistentDispatcherMultipleConsumers class #17018

Metadata & Storage​

  • [fix][metadata] Fix LockTimeout when storePut on the same key concurrently in RocksdbMetadataStore #16005
  • [fix][metadata] Cleanup state when lock revalidation gets LockBusyException #17700
  • [fix][metadata] Don't execute Bookkeeper metadata callbacks on Zookeeper event thread #17620
  • [fix][metadata] Set revalidateAfterReconnection true for certain failures #17664
  • [improve][metadata] Update namespace policies would cause metadata-store thread waiting too long #16438
  • [fix][storage] Fix offload read handle NPE. #17478
  • [fix][storage] Fix OpAddEntry release error when exception in ManagedLedgerInterceptor #17394
  • [fix][tableview] Fixed ack failure in ReaderImpl due to null messageId #17828

Transaction​

  • [fix][txn] Cmd-Subscribe and Cmd-Producer will not succeed even after 100 retries #16248
  • [fix][txn] Set TC state is Ready after open MLTransactionMetadataStore completely. #13957
  • [fix][txn] Ack the same batch message different batchIndex with transaction #16032
  • [fix][txn] Allow producer to enable send timeout in transaction #16519
  • [fix][txn] Fix NPE when ack message with transaction at cnx = null #16142
  • [fix][txn] Fix TopicTransactionBuffer ledger apend marker throw ManagedLedgerAlreadyClosedException #16265
  • [fix][txn] Fix append txn message is lower than lowWaterMark decrease PendingWriteOps #16266
  • [fix][txn] Fix ack with txn compute ackedCount error #17016
  • [fix][txn] Fix pattern sub filter transaction system topic #16533
  • [fix][txn] Fix race in TransactionMetaStoreHandler #16147

Pulsar IO and Pulsar Functions​

  • [fix][connector] Fix Kafka source config for pulsar-io-kafka when consumerConfigProperties="" #16731
  • [fix][connector] IOConfigUtils support required and defaultValue annotations. #16785
  • [improve][connector] Add reader config to pulsar-io-debezium and pulsar-io-kafka-connect-adaptor #16675
  • [fix][function] Ensure bytes is a well-formed UTF-8 byte sequence when decode the FunctionState bytes to string #16199
  • [fix][function] Fix python instance not process zip file correctly #16697
  • [fix][function] Fixed error when user starts with the pulsar functions local runner #16565
  • [fix][function] Ensure InternalConfigurationData data model is compatible across different versions #17690

Tiered Storage​

  • [fix][tiered-storage] Don't cleanup data when offload met Metastore exception #17512
  • [fix][tiered-storage] Fix the wrong secret key name get from env #15814
  • [fix][tiered-storage] move the state check forward #17020
  • [fix][common] Fix setManagedLedgerOffloadedReadPriority not work. #16436

Pulsar SQL​

  • [fix][sql] Fix messageQueue release message issue. #16155
  • [fix][sql] Fix Presto SQL Avro decode error when publishing non-batched msgs #17093

Pulsar Proxy​

  • [fix][proxy] Do not preserve host when forwarding admin requests. #16342
  • [fix][proxy] Fix client service URL #16834
  • [improve][proxy] Update proxy lookup throw exception type #17600

CLI/Admin​

  • [fix][admin] Fix reach max tenants error if the tenant already exists #15932
  • [fix][admin] Fix get non-persistent topics issue in Namespaces. #16514
  • [fix][admin] Add SNI header when tlsHostnameVerification is not enabled #17543
  • [fix][admin] Fix missing response type in swagger definitions #16022
  • [fix][admin] Fix typo in validation message #16021
  • [fix][admin] Fix unWrap Exception when getPoliciesAsync #17249
  • [cleanup][admin] Update/fix Swagger Annotation for param: authoritative #16222

Security​

  • [fix][sec] Avoid AuthenticationDataSource mutation for subscription name #16065
  • [fix][sec] Upgrade Netty Reactive Streams to 2.0.6 #15990
  • [fix][sec] Bump dependency check and spring version to avoid potential FP #15408
  • [fix][sec] Bump snakeyaml to 1.32 for CVE-2022-38752 #17779
  • [fix][sec] Bump snakeyaml to 1.31 fix CVE-2022-25857 #17457
  • [fix][sec] Bump PostgreSQL version to 42.4.1 #17066
  • [fix][sec] Upgrade reload4j in file-system offloader #17716
  • [fix][sec] Upgrade to Jetty to 9.4.48.v20220622 to get rid of CVE-2022-2047 #16520
  • [improve][sec] Suppress CVE-2021-3563 of openstack-keystone-2.5.0 #17458
  • [improve][sec] Add load multiple certificates support #14798
  • [improve][sec] Upgrade aws-java-sdk-s3 to 1.12.261 #16684
  • [fix][authorization] Fix multiple roles authorization #16645
  • [improve][authentication] Adapt basic authentication configuration with prefix #16935
  • [improve][authentication] Improve get the basic authentication config #16526
  • [improve][authentication] Support for get token from HTTP params #16986

Dependency & Library updates​

  • [fix][dependency] Exclude the Netty Reactive Stream from asynchttpclient #16312
  • [fix][dependency] Removing log4j-1.2-api from dependencies #15991
  • [cleanup][dependency] Remove redundant pulsar-zookeeper-utils module #16258

CI & Test​

  • [fix][ci] Remove post-commit trigger for old release branches (2.10 and previous) #17570
  • [fix][ci] Fix some OWASP dependency problems. #16260
  • [fix][test] Fix flaky-test RackAwareTest.testRackUpdate #16071
  • [fix][test] Catch exception when updating data in mockZookeeper #16473
  • [fix][test] Fix jvm oom on Unit Test broker group 1 #16542
  • [fix][test] Fix ManagedLedgerTest#avoidUseSameOpAddEntryBetweenDifferentLedger #16720
  • [fix][test] Fix test TransactionEndToEndTest#testSendTxnMessageTimeout (only release branches) #16570
  • [fix][test] Fix Flaky-test: PartitionedProducerConsumerTest.testPartitionedTopicInterceptor #16809
  • [fix][test] TieredStorageConfigurationTests - clear system properties #15957
  • [fix][test] Fix Flaky-test: ManagedLedgerBkTest.asyncMarkDeleteAndClose. #16730
  • [fix][test] Fix flaky C++ ClientTest.testWrongListener #16510
  • [improve][test] Reduce the time consumption of BacklogQuotaManagerTest #16550
  • [improve][test] Verify the authentication data in the authorization provider #16900
  • [improve][test] Improved flaky test runs #16011
  • [fix][flaky-test] BrokerInterceptorTest.testProducerCreation #16742
  • [fix][flaky-test] ElasticSearchClientTests.testBulkBlocking #16920
  • [fix][flaky-test] Fix BacklogQuotaManagerTest.testConsumerBacklogEvictionTimeQuotaWithEmptyLedge #16735
  • [fix][flaky-test] Fix ClassCastException: BrokerService cannot be cast to class PulsarResources #16821
  • [fix][flaky-test] Fix DefaultMessageFormatter.formatMessage #17104
  • [fix][flaky-test] Fix PersistentTopicStreamingDispatcherTest #16801
  • [fix][flaky-test] Fix PulsarFunctionTlsTest.tearDown #16765
  • [fix][flaky-test] Fix failed test NonPersistentTopicE2ETest.testGCWillDeleteSchema #16381
  • [fix][flaky-test] Fix failed test PatternTopicsConsumerImplTest.testAutoSubscribePatternConsumer #16375
  • [fix][flaky-test] Fix flaky test testBacklogNoDelayedForPartitionedTopic #17180
  • [fix][flaky-test] Fix flaky test testConsumerBacklogEvictionTimeQuota… #16419
  • [fix][flaky-test] MessageTTLTest.testMessageExpiryAfterTopicUnload #16462
  • [fix][flaky-test] NonPersistentTopicE2ETest.testGC #16505
  • [fix][flaky-test] PersistentFailoverE2ETest.testSimpleConsumerEventsWithPartition #16493
  • [fix][flaky-test] Try to fix flaky test related to PersistentTopicTest.setup #16383
  • [fix][flaky-test] Fix testSplitBundleForMultiTimes #16562
  • [fix][flaky-test] PulsarFunctionLocalRunTest.testE2EPulsarFunctionLocalRunMultipleInstances #16872
  • [fix][flaky-test] AdminApi2Test.testDeleteNamespace #17157
  • [fix][flaky-test] BrokerInterceptorTest.testProducerCreation #17159
  • [fix][flaky-test] ConsumedLedgersTrimTest #17116
  • [fix][flaky-test] ManagedCursorMetricsTest.testCursorReadWriteMetrics #17045

Others​

  • [fix][build] Duplicate entry when merging services #17659
  • [cleanup][owasp] Suppress false positive netty-tcnative #17282
  • [fix][doc] Fix comments for exposeManagedLedgerMetricsInPrometheus field #17792
  • [fix][tool] Using int instead of long in python scripts #17215
  • [improve][package] Improve the package download process #16365
Was this helpful?
💡 Suggest changes🛟 Get support