pulsar.source.autoCommitCursorInterval |
5000 |
Long |
This option is used only when the user disables the checkpoint and uses Exclusive or Failover subscription. We would automatically commit the cursor using the given period (in ms). |
pulsar.source.enableAutoAcknowledgeMessage |
false |
Boolean |
Flink commits the consuming position with pulsar transactions on checkpoint. However, if you have disabled the Flink checkpoint or disabled transaction for your Pulsar cluster, ensure that you have set this option to true . The source would use pulsar client's internal mechanism and commit cursor in two ways.- For
Key_Shared and Shared subscription, the cursor would be committed once the message is consumed. - For
Exclusive and Failover subscription, the cursor would be committed in a given interval.
|
pulsar.source.maxFetchRecords |
100 |
Integer |
The maximum number of records to fetch to wait when polling. A longer time increases throughput but also latency. A fetch batch might be finished earlier because of pulsar.source.maxFetchTime . |
pulsar.source.maxFetchTime |
10000 |
Long |
The maximum time (in ms) to wait when fetching records. A longer time increases throughput but also latency. A fetch batch might be finished earlier because of pulsar.source.maxFetchRecords . |
pulsar.source.partitionDiscoveryIntervalMs |
30000 |
Long |
The interval (in ms) for the Pulsar source to discover the new partitions. A non-positive value disables the partition discovery. |
pulsar.source.transactionTimeoutMillis |
10800000 |
Long |
This option is used in Shared or Key_Shared subscription. You should configure this option when you do not enable the pulsar.source.enableAutoAcknowledgeMessage option. The value (in ms) should be greater than the checkpoint interval. |
pulsar.source.verifyInitialOffsets |
WARN_ON_MISMATCH |
Enum |
Upon (re)starting the source, check whether the expected message can be read. If failure is enabled, the application fails. Otherwise, it logs a warning. A possible solution is to adjust the retention settings in Pulsar or ignoring the check result.
Possible values:- "FAIL_ON_MISMATCH"
- "WARN_ON_MISMATCH"
|