task_manager_configuration.html

<table class="configuration table table-bordered">
    <thead>
        <tr>
            <th class="text-left" style="width: 20%">Key</th>
            <th class="text-left" style="width: 15%">Default</th>
            <th class="text-left" style="width: 10%">Type</th>
            <th class="text-left" style="width: 55%">Description</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td><h5>minicluster.number-of-taskmanagers</h5></td>
            <td style="word-wrap: break-word;">1</td>
            <td>Integer</td>
            <td>The number of task managers of MiniCluster.</td>
        </tr>
        <tr>
            <td><h5>task.cancellation.interval</h5></td>
            <td style="word-wrap: break-word;">30 s</td>
            <td>Duration</td>
            <td>Time interval between two successive task cancellation attempts.</td>
        </tr>
        <tr>
            <td><h5>task.cancellation.timeout</h5></td>
            <td style="word-wrap: break-word;">3 min</td>
            <td>Duration</td>
            <td>Timeout after which a task cancellation times out and leads to a fatal TaskManager error. A value of 0 deactivates the watch dog. Notice that a task cancellation is different from both a task failure and a clean shutdown.  Task cancellation timeout only applies to task cancellation and does not apply to task closing/clean-up caused by a task failure or a clean shutdown.</td>
        </tr>
        <tr>
            <td><h5>task.cancellation.timers.timeout</h5></td>
            <td style="word-wrap: break-word;">7500 ms</td>
            <td>Duration</td>
            <td>Time we wait for the timers to finish all pending timer threads when the stream task is cancelled.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.bind-host</h5></td>
            <td style="word-wrap: break-word;">(none)</td>
            <td>String</td>
            <td>The local address of the network interface that the task manager binds to. If not configured, '0.0.0.0' will be used.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.collect-sink.port</h5></td>
            <td style="word-wrap: break-word;">0</td>
            <td>Integer</td>
            <td>The port used for the client to retrieve query results from the TaskManager. The default value is 0, which corresponds to a random port assignment.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.debug.memory.log</h5></td>
            <td style="word-wrap: break-word;">false</td>
            <td>Boolean</td>
            <td>Flag indicating whether to start a thread, which repeatedly logs the memory usage of the JVM.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.debug.memory.log-interval</h5></td>
            <td style="word-wrap: break-word;">5 s</td>
            <td>Duration</td>
            <td>The interval for the log thread to log the current memory usage.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.host</h5></td>
            <td style="word-wrap: break-word;">(none)</td>
            <td>String</td>
            <td>The external address of the network interface where the TaskManager is exposed. Because different TaskManagers need different values for this option, usually it is specified in an additional non-shared TaskManager-specific config file.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.jvm-exit-on-oom</h5></td>
            <td style="word-wrap: break-word;">false</td>
            <td>Boolean</td>
            <td>Whether to kill the TaskManager when the task thread throws an OutOfMemoryError.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.load-balance.mode</h5></td>
            <td style="word-wrap: break-word;">NONE</td>
            <td><p>Enum</p></td>
            <td>Mode for the load-balance allocation strategy across all available <code class="highlighter-rouge">TaskManagers</code>.<ul><li>The <code class="highlighter-rouge">SLOTS</code> mode tries to spread out the slots evenly across all available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">MIN_RESOURCES</code> mode tries to allocate slots on minimum number of available <code class="highlighter-rouge">TaskManagers</code>.</li><li>The <code class="highlighter-rouge">NONE</code> mode is the default mode without any specified strategy.</li></ul><br /><br />Possible values:<ul><li>"NONE"</li><li>"SLOTS"</li><li>"MIN_RESOURCES"</li></ul></td>
        </tr>
        <tr>
            <td><h5>taskmanager.log.path</h5></td>
            <td style="word-wrap: break-word;">System.getProperty("log.file")</td>
            <td>String</td>
            <td>The path to the log file of the task manager.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.network.bind-policy</h5></td>
            <td style="word-wrap: break-word;">"ip"</td>
            <td>String</td>
            <td>The automatic address binding policy used by the TaskManager if "taskmanager.host" is not set. The value should be one of the following:
<ul><li>"name" - uses hostname as binding address</li><li>"ip" - uses host's ip address as binding address</li></ul></td>
        </tr>
        <tr>
            <td><h5>taskmanager.network.memory.buffer-debloat.enabled</h5></td>
            <td style="word-wrap: break-word;">false</td>
            <td>Boolean</td>
            <td>The switch of the automatic buffered debloating feature. If enabled the amount of in-flight data will be adjusted automatically accordingly to the measured throughput.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.network.memory.buffer-debloat.period</h5></td>
            <td style="word-wrap: break-word;">200 ms</td>
            <td>Duration</td>
            <td>The minimum period of time after which the buffer size will be debloated if required. The low value provides a fast reaction to the load fluctuation but can influence the performance.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.network.memory.buffer-debloat.samples</h5></td>
            <td style="word-wrap: break-word;">20</td>
            <td>Integer</td>
            <td>The number of the last buffer size values that will be taken for the correct calculation of the new one.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.network.memory.buffer-debloat.target</h5></td>
            <td style="word-wrap: break-word;">1 s</td>
            <td>Duration</td>
            <td>The target total time after which buffered in-flight data should be fully consumed. This configuration option will be used, in combination with the measured throughput, to adjust the amount of in-flight data.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.network.memory.buffer-debloat.threshold-percentages</h5></td>
            <td style="word-wrap: break-word;">25</td>
            <td>Integer</td>
            <td>The minimum difference in percentage between the newly calculated buffer size and the old one to announce the new value. Can be used to avoid constant back and forth small adjustments.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.numberOfTaskSlots</h5></td>
            <td style="word-wrap: break-word;">1</td>
            <td>Integer</td>
            <td>The number of parallel operator or user function instances that a single TaskManager can run. If this value is larger than 1, a single TaskManager takes multiple instances of a function or operator. That way, the TaskManager can utilize multiple CPU cores, but at the same time, the available memory is divided between the different operator or function instances. This value is typically proportional to the number of physical CPU cores that the TaskManager's machine has (e.g., equal to the number of cores, or half the number of cores).</td>
        </tr>
        <tr>
            <td><h5>taskmanager.registration.timeout</h5></td>
            <td style="word-wrap: break-word;">5 min</td>
            <td>Duration</td>
            <td>Defines the timeout for the TaskManager registration. If the duration is exceeded without a successful registration, then the TaskManager terminates.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.resource-id</h5></td>
            <td style="word-wrap: break-word;">(none)</td>
            <td>String</td>
            <td>The TaskManager's ResourceID. If not configured, the ResourceID will be generated with the "RpcAddress:RpcPort" and a 6-character random string. Notice that this option is not valid in Yarn and Native Kubernetes mode.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.rpc.bind-port</h5></td>
            <td style="word-wrap: break-word;">(none)</td>
            <td>Integer</td>
            <td>The local RPC port that the TaskManager binds to. If not configured, the external port (configured by 'taskmanager.rpc.port') will be used.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.rpc.port</h5></td>
            <td style="word-wrap: break-word;">"0"</td>
            <td>String</td>
            <td>The external RPC port where the TaskManager is exposed. Accepts a list of ports (“50100,50101”), ranges (“50100-50200”) or a combination of both. It is recommended to set a range of ports to avoid collisions when multiple TaskManagers are running on the same machine.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.runtime.fs-timeout</h5></td>
            <td style="word-wrap: break-word;">0 ms</td>
            <td>Duration</td>
            <td>The timeout for filesystem stream opening. A value of 0 indicates infinite waiting.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.slot.timeout</h5></td>
            <td style="word-wrap: break-word;">10 s</td>
            <td>Duration</td>
            <td>Timeout used for identifying inactive slots. The TaskManager will free the slot if it does not become active within the given amount of time. Inactive slots can be caused by an out-dated slot request. If no value is configured, then it will fall back to <code class="highlighter-rouge">pekko.ask.timeout</code>.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.system-out.log.cache-upper-size</h5></td>
            <td style="word-wrap: break-word;">100 kb</td>
            <td>MemorySize</td>
            <td>The cache upper size when Flink caches current line context of <code class="highlighter-rouge">System.out</code> or <code class="highlighter-rouge">System.err</code> when <code class="highlighter-rouge">taskmanager.system-out.mode</code> is LOG.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.system-out.log.thread-name.enabled</h5></td>
            <td style="word-wrap: break-word;">false</td>
            <td>Boolean</td>
            <td>Whether to log the thread name when <code class="highlighter-rouge">taskmanager.system-out.mode</code> is LOG.</td>
        </tr>
        <tr>
            <td><h5>taskmanager.system-out.mode</h5></td>
            <td style="word-wrap: break-word;">DEFAULT</td>
            <td><p>Enum</p></td>
            <td>Redirection mode of <code class="highlighter-rouge">System.out</code> and <code class="highlighter-rouge">System.err</code> for all <code class="highlighter-rouge">TaskManagers</code>.<ul><li><code class="highlighter-rouge">DEFAULT</code>: <code class="highlighter-rouge">TaskManagers</code> don't redirect the <code class="highlighter-rouge">System.out</code> and <code class="highlighter-rouge">System.err</code>, it's the default value.</li><li><code class="highlighter-rouge">LOG</code>: <code class="highlighter-rouge">TaskManagers</code> redirect <code class="highlighter-rouge">System.out</code> and <code class="highlighter-rouge">System.err</code> to LOG.info and LOG.error.</li><li><code class="highlighter-rouge">IGNORE</code>: <code class="highlighter-rouge">TaskManagers</code> ignore <code class="highlighter-rouge">System.out</code> and <code class="highlighter-rouge">System.err</code> directly.</li></ul><br /><br />Possible values:<ul><li>"DEFAULT"</li><li>"LOG"</li><li>"IGNORE"</li></ul></td>
        </tr>
    </tbody>
</table>