Skip to content

Commit 22305cf

Browse files
zlzhang0122zlzhang0122
committed
data comment
1 parent 91e4ab1 commit 22305cf

12 files changed

+100
-46
lines changed

‎.idea/workspace.xml

+56-27
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/_build/doctrees/dataDeliver.doctree

1.98 KB
Binary file not shown.

‎docs/_build/doctrees/environment.pickle

-76 Bytes
Binary file not shown.

‎docs/_build/html/_images/shuffle.png

578 KB
Loading

‎docs/_build/html/_sources/dataDeliver.rst.txt

+9-3
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,16 @@
22
===============
33
本节主要介绍数据在各个节点之间是如何进行传递的,以及如何在数据传递的过程中进行自然的反压.
44

5-
Subject Subtitle
5+
数据传递流程
66
----------------
7-
Subtitles are set with '-' and are required to have the same length
8-
of the subtitle itself, just like titles.
7+
.. image:: image/shuffle.png
8+
9+
整体实现步骤如下:
10+
* 在M1/M2处理完数据后,本地需要ResultPartition RS1/RS2来临时存储数据;
11+
* 通知JobManager,上游有新的数据产生;
12+
* JobManager通知和调度下游节点可以消费新的数据;
13+
* 下游节点向上游请求数据;
14+
* 通过Channel在各个TaskManager之间传递数据.
915

1016
Lists can be unnumbered like:
1117

‎docs/_build/html/contents.html

+1-1
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ <h1>Flink源码阅读文档!<a class="headerlink" href="#flink" title="Permalink
158158
</ul>
159159
</li>
160160
<li class="toctree-l1"><a class="reference internal" href="dataDeliver.html">数据传递</a><ul>
161-
<li class="toctree-l2"><a class="reference internal" href="dataDeliver.html#subject-subtitle">Subject Subtitle</a></li>
161+
<li class="toctree-l2"><a class="reference internal" href="dataDeliver.html#id2">数据传递流程</a></li>
162162
</ul>
163163
</li>
164164
</ul>

‎docs/_build/html/dataDeliver.html

+16-5
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@
8383
<ul class="current">
8484
<li class="toctree-l1"><a class="reference internal" href="jobSubmit.html">任务提交</a></li>
8585
<li class="toctree-l1 current"><a class="current reference internal" href="#">数据传递</a><ul>
86-
<li class="toctree-l2"><a class="reference internal" href="#subject-subtitle">Subject Subtitle</a></li>
86+
<li class="toctree-l2"><a class="reference internal" href="#id2">数据传递流程</a></li>
8787
</ul>
8888
</li>
8989
</ul>
@@ -153,10 +153,21 @@
153153
<div class="section" id="id1">
154154
<h1>数据传递<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h1>
155155
<p>本节主要介绍数据在各个节点之间是如何进行传递的,以及如何在数据传递的过程中进行自然的反压.</p>
156-
<div class="section" id="subject-subtitle">
157-
<h2>Subject Subtitle<a class="headerlink" href="#subject-subtitle" title="Permalink to this headline"></a></h2>
158-
<p>Subtitles are set with ‘-‘ and are required to have the same length
159-
of the subtitle itself, just like titles.</p>
156+
<div class="section" id="id2">
157+
<h2>数据传递流程<a class="headerlink" href="#id2" title="Permalink to this headline"></a></h2>
158+
<blockquote>
159+
<div><img alt="_images/shuffle.png" src="_images/shuffle.png" />
160+
</div></blockquote>
161+
<dl class="simple">
162+
<dt>整体实现步骤如下:</dt><dd><ul class="simple">
163+
<li><p>在M1/M2处理完数据后,本地需要ResultPartition RS1/RS2来临时存储数据;</p></li>
164+
<li><p>通知JobManager,上游有新的数据产生;</p></li>
165+
<li><p>JobManager通知和调度下游节点可以消费新的数据;</p></li>
166+
<li><p>下游节点向上游请求数据;</p></li>
167+
<li><p>通过Channel在各个TaskManager之间传递数据.</p></li>
168+
</ul>
169+
</dd>
170+
</dl>
160171
<p>Lists can be unnumbered like:</p>
161172
<blockquote>
162173
<div><ul class="simple">

‎docs/_build/html/example.html

+4-3
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@
4848

4949

5050

51-
<a href="index.html" class="icon icon-home"> flink源码阅读
51+
<a href="contents.html" class="icon icon-home"> flink源码阅读
5252

5353

5454

@@ -81,6 +81,7 @@
8181
<p class="caption"><span class="caption-text">Contents:</span></p>
8282
<ul>
8383
<li class="toctree-l1"><a class="reference internal" href="jobSubmit.html">任务提交</a></li>
84+
<li class="toctree-l1"><a class="reference internal" href="dataDeliver.html">数据传递</a></li>
8485
</ul>
8586

8687

@@ -95,7 +96,7 @@
9596
<nav class="wy-nav-top" aria-label="top navigation">
9697

9798
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
98-
<a href="index.html">flink源码阅读</a>
99+
<a href="contents.html">flink源码阅读</a>
99100

100101
</nav>
101102

@@ -124,7 +125,7 @@
124125

125126
<ul class="wy-breadcrumbs">
126127

127-
<li><a href="index.html">Docs</a> &raquo;</li>
128+
<li><a href="contents.html">Docs</a> &raquo;</li>
128129

129130
<li>This is a Title</li>
130131

‎docs/_build/html/jobSubmit.html

+9-5
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,8 @@
3535
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
3636
<link rel="index" title="Index" href="genindex.html" />
3737
<link rel="search" title="Search" href="search.html" />
38-
<link rel="prev" title="Flink源码阅读文档!" href="index.html" />
38+
<link rel="next" title="数据传递" href="dataDeliver.html" />
39+
<link rel="prev" title="Flink源码阅读文档!" href="contents.html" />
3940
</head>
4041

4142
<body class="wy-body-for-nav">
@@ -49,7 +50,7 @@
4950

5051

5152

52-
<a href="index.html" class="icon icon-home"> flink源码阅读
53+
<a href="contents.html" class="icon icon-home"> flink源码阅读
5354

5455

5556

@@ -86,6 +87,7 @@
8687
<li class="toctree-l2"><a class="reference internal" href="#client">Client</a></li>
8788
</ul>
8889
</li>
90+
<li class="toctree-l1"><a class="reference internal" href="dataDeliver.html">数据传递</a></li>
8991
</ul>
9092

9193

@@ -100,7 +102,7 @@
100102
<nav class="wy-nav-top" aria-label="top navigation">
101103

102104
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
103-
<a href="index.html">flink源码阅读</a>
105+
<a href="contents.html">flink源码阅读</a>
104106

105107
</nav>
106108

@@ -129,7 +131,7 @@
129131

130132
<ul class="wy-breadcrumbs">
131133

132-
<li><a href="index.html">Docs</a> &raquo;</li>
134+
<li><a href="contents.html">Docs</a> &raquo;</li>
133135

134136
<li>任务提交</li>
135137

@@ -232,8 +234,10 @@ <h2>Client<a class="headerlink" href="#client" title="Permalink to this headline
232234

233235
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
234236

237+
<a href="dataDeliver.html" class="btn btn-neutral float-right" title="数据传递" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
235238

236-
<a href="index.html" class="btn btn-neutral float-left" title="Flink源码阅读文档!" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
239+
240+
<a href="contents.html" class="btn btn-neutral float-left" title="Flink源码阅读文档!" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
237241

238242
</div>
239243

0 commit comments

Comments
 (0)