Skip to content

Commit 4db36c2

Browse files
shenweichenshenweichen
authored
add EDCN model
- add EDCN model - fix savedmodel error in tf2.4 - fix typo in position encoding layer
2 parents ec78b9b + 6789597 commit 4db36c2

25 files changed

+491
-132
lines changed

‎.github/workflows/ci.yml

+33-33
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
name: CI
1+
name: CI_TF2
22

3-
on:
3+
on:
44
push:
55
path:
66
- 'deepctr/*'
@@ -9,17 +9,17 @@ on:
99
path:
1010
- 'deepctr/*'
1111
- 'tests/*'
12-
12+
1313
jobs:
1414
build:
1515

1616
runs-on: ubuntu-latest
1717
timeout-minutes: 180
1818
strategy:
1919
matrix:
20-
python-version: [3.6,3.7,3.8,3.9,3.10.7]
21-
tf-version: [1.4.0,1.15.0,2.6.0,2.7.0,2.8.0,2.9.0,2.10.0]
22-
20+
python-version: [ 3.6,3.7,3.8, 3.9,3.10.7 ]
21+
tf-version: [ 2.6.0,2.7.0,2.8.0,2.9.0,2.10.0 ]
22+
2323
exclude:
2424
- python-version: 3.7
2525
tf-version: 1.4.0
@@ -64,31 +64,31 @@ jobs:
6464
- python-version: 3.10.7
6565
tf-version: 2.7.0
6666
steps:
67-
68-
- uses: actions/checkout@v3
69-
70-
- name: Setup python environment
71-
uses: actions/setup-python@v4
72-
with:
73-
python-version: ${{ matrix.python-version }}
7467

75-
- name: Install dependencies
76-
run: |
77-
pip3 install -q tensorflow==${{ matrix.tf-version }}
78-
pip install -q protobuf==3.19.0
79-
pip install -q requests
80-
pip install -e .
81-
- name: Test with pytest
82-
timeout-minutes: 180
83-
run: |
84-
pip install -q pytest
85-
pip install -q pytest-cov
86-
pip install -q python-coveralls
87-
pytest --cov=deepctr --cov-report=xml
88-
- name: Upload coverage to Codecov
89-
uses: codecov/[email protected]
90-
with:
91-
token: ${{secrets.CODECOV_TOKEN}}
92-
file: ./coverage.xml
93-
flags: pytest
94-
name: py${{ matrix.python-version }}-tf${{ matrix.tf-version }}
68+
- uses: actions/checkout@v3
69+
70+
- name: Setup python environment
71+
uses: actions/setup-python@v4
72+
with:
73+
python-version: ${{ matrix.python-version }}
74+
75+
- name: Install dependencies
76+
run: |
77+
pip3 install -q tensorflow==${{ matrix.tf-version }}
78+
pip install -q protobuf==3.19.0
79+
pip install -q requests
80+
pip install -e .
81+
- name: Test with pytest
82+
timeout-minutes: 180
83+
run: |
84+
pip install -q pytest
85+
pip install -q pytest-cov
86+
pip install -q python-coveralls
87+
pytest --cov=deepctr --cov-report=xml
88+
- name: Upload coverage to Codecov
89+
uses: codecov/[email protected]
90+
with:
91+
token: ${{secrets.CODECOV_TOKEN}}
92+
file: ./coverage.xml
93+
flags: pytest
94+
name: py${{ matrix.python-version }}-tf${{ matrix.tf-version }}

‎.github/workflows/ci2.yml

+96
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
name: CI_TF1
2+
3+
on:
4+
push:
5+
path:
6+
- 'deepctr/*'
7+
- 'tests/*'
8+
pull_request:
9+
path:
10+
- 'deepctr/*'
11+
- 'tests/*'
12+
13+
jobs:
14+
build:
15+
16+
runs-on: ubuntu-latest
17+
timeout-minutes: 180
18+
strategy:
19+
matrix:
20+
python-version: [ 3.6,3.7 ]
21+
tf-version: [ 1.15.0 ]
22+
23+
exclude:
24+
- python-version: 3.7
25+
tf-version: 1.4.0
26+
- python-version: 3.7
27+
tf-version: 1.12.0
28+
- python-version: 3.7
29+
tf-version: 1.15.0
30+
- python-version: 3.8
31+
tf-version: 1.4.0
32+
- python-version: 3.8
33+
tf-version: 1.14.0
34+
- python-version: 3.8
35+
tf-version: 1.15.0
36+
- python-version: 3.6
37+
tf-version: 2.7.0
38+
- python-version: 3.6
39+
tf-version: 2.8.0
40+
- python-version: 3.6
41+
tf-version: 2.9.0
42+
- python-version: 3.6
43+
tf-version: 2.10.0
44+
- python-version: 3.9
45+
tf-version: 1.4.0
46+
- python-version: 3.9
47+
tf-version: 1.15.0
48+
- python-version: 3.9
49+
tf-version: 2.2.0
50+
- python-version: 3.9
51+
tf-version: 2.5.0
52+
- python-version: 3.9
53+
tf-version: 2.6.0
54+
- python-version: 3.9
55+
tf-version: 2.7.0
56+
- python-version: 3.10.7
57+
tf-version: 1.4.0
58+
- python-version: 3.10.7
59+
tf-version: 1.15.0
60+
- python-version: 3.10.7
61+
tf-version: 2.2.0
62+
- python-version: 3.10.7
63+
tf-version: 2.5.0
64+
- python-version: 3.10.7
65+
tf-version: 2.6.0
66+
- python-version: 3.10.7
67+
tf-version: 2.7.0
68+
steps:
69+
70+
- uses: actions/checkout@v3
71+
72+
- name: Setup python environment
73+
uses: actions/setup-python@v4
74+
with:
75+
python-version: ${{ matrix.python-version }}
76+
77+
- name: Install dependencies
78+
run: |
79+
pip3 install -q tensorflow==${{ matrix.tf-version }}
80+
pip install -q protobuf==3.19.0
81+
pip install -q requests
82+
pip install -e .
83+
- name: Test with pytest
84+
timeout-minutes: 180
85+
run: |
86+
pip install -q pytest
87+
pip install -q pytest-cov
88+
pip install -q python-coveralls
89+
pytest --cov=deepctr --cov-report=xml
90+
- name: Upload coverage to Codecov
91+
uses: codecov/[email protected]
92+
with:
93+
token: ${{secrets.CODECOV_TOKEN}}
94+
file: ./coverage.xml
95+
flags: pytest
96+
name: py${{ matrix.python-version }}-tf${{ matrix.tf-version }}

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ Introduction](https://zhuanlan.zhihu.com/p/53231955)) and [welcome to join us!](
6666
| ESMM | [SIGIR 2018][Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://arxiv.org/abs/1804.07931) |
6767
| MMOE | [KDD 2018][Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts](https://dl.acm.org/doi/abs/10.1145/3219819.3220007) |
6868
| PLE | [RecSys 2020][Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/10.1145/3383313.3412236) |
69+
| EDCN | [KDD 2021][Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf) |
6970

7071
## Citation
7172

‎deepctr/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
from .utils import check_version
22

3-
__version__ = '0.9.2'
3+
__version__ = '0.9.3'
44
check_version(__version__)

‎deepctr/layers/__init__.py

+8-6
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,16 @@
11
import tensorflow as tf
22

33
from .activation import Dice
4-
from .core import DNN, LocalActivationUnit, PredictionLayer
4+
from .core import DNN, LocalActivationUnit, PredictionLayer, RegulationModule
55
from .interaction import (CIN, FM, AFMLayer, BiInteractionPooling, CrossNet, CrossNetMix,
66
InnerProductLayer, InteractingLayer,
77
OutterProductLayer, FGCNNLayer, SENETLayer, BilinearInteraction,
8-
FieldWiseBiInteraction, FwFMLayer, FEFMLayer)
8+
FieldWiseBiInteraction, FwFMLayer, FEFMLayer, BridgeModule)
99
from .normalization import LayerNormalization
1010
from .sequence import (AttentionSequencePoolingLayer, BiasEncoding, BiLSTM,
1111
KMaxPooling, SequencePoolingLayer, WeightedSequenceLayer,
12-
Transformer, DynamicGRU,PositionEncoding)
13-
14-
from .utils import NoMask, Hash, Linear, _Add, combined_dnn_input, softmax, reduce_sum
12+
Transformer, DynamicGRU, PositionEncoding)
13+
from .utils import NoMask, Hash, Linear, _Add, combined_dnn_input, softmax, reduce_sum, Concat
1514

1615
custom_objects = {'tf': tf,
1716
'InnerProductLayer': InnerProductLayer,
@@ -38,6 +37,7 @@
3837
'FGCNNLayer': FGCNNLayer,
3938
'Hash': Hash,
4039
'Linear': Linear,
40+
'Concat': Concat,
4141
'DynamicGRU': DynamicGRU,
4242
'SENETLayer': SENETLayer,
4343
'BilinearInteraction': BilinearInteraction,
@@ -48,5 +48,7 @@
4848
'softmax': softmax,
4949
'FEFMLayer': FEFMLayer,
5050
'reduce_sum': reduce_sum,
51-
'PositionEncoding':PositionEncoding
51+
'PositionEncoding': PositionEncoding,
52+
'RegulationModule': RegulationModule,
53+
'BridgeModule': BridgeModule
5254
}

‎deepctr/layers/core.py

+56-2
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@
1010
from tensorflow.python.keras import backend as K
1111

1212
try:
13-
from tensorflow.python.ops.init_ops_v2 import Zeros, glorot_normal
13+
from tensorflow.python.ops.init_ops_v2 import Zeros, Ones, glorot_normal
1414
except ImportError:
15-
from tensorflow.python.ops.init_ops import Zeros, glorot_normal_initializer as glorot_normal
15+
from tensorflow.python.ops.init_ops import Zeros, Ones, glorot_normal_initializer as glorot_normal
1616

1717
from tensorflow.python.keras.layers import Layer, Dropout
1818

@@ -265,3 +265,57 @@ def get_config(self, ):
265265
config = {'task': self.task, 'use_bias': self.use_bias}
266266
base_config = super(PredictionLayer, self).get_config()
267267
return dict(list(base_config.items()) + list(config.items()))
268+
269+
270+
class RegulationModule(Layer):
271+
"""Regulation module used in EDCN.
272+
273+
Input shape
274+
- 3D tensor with shape: ``(batch_size,field_size,embedding_size)``.
275+
276+
Output shape
277+
- 2D tensor with shape: ``(batch_size,field_size * embedding_size)``.
278+
279+
Arguments
280+
- **tau** : Positive float, the temperature coefficient to control
281+
distribution of field-wise gating unit.
282+
283+
References
284+
- [Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models.](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf)
285+
"""
286+
287+
def __init__(self, tau=1.0, **kwargs):
288+
if tau == 0:
289+
raise ValueError("RegulationModule tau can not be zero.")
290+
self.tau = 1.0 / tau
291+
super(RegulationModule, self).__init__(**kwargs)
292+
293+
def build(self, input_shape):
294+
self.field_size = int(input_shape[1])
295+
self.embedding_size = int(input_shape[2])
296+
self.g = self.add_weight(
297+
shape=(1, self.field_size, 1),
298+
initializer=Ones(),
299+
name=self.name + '_field_weight')
300+
301+
# Be sure to call this somewhere!
302+
super(RegulationModule, self).build(input_shape)
303+
304+
def call(self, inputs, **kwargs):
305+
306+
if K.ndim(inputs) != 3:
307+
raise ValueError(
308+
"Unexpected inputs dimensions %d, expect to be 3 dimensions" % (K.ndim(inputs)))
309+
310+
feild_gating_score = tf.nn.softmax(self.g * self.tau, 1)
311+
E = inputs * feild_gating_score
312+
return tf.reshape(E, [-1, self.field_size * self.embedding_size])
313+
314+
def compute_output_shape(self, input_shape):
315+
return (None, self.field_size * self.embedding_size)
316+
317+
def get_config(self):
318+
config = {'tau': self.tau}
319+
base_config = super(RegulationModule, self).get_config()
320+
base_config.update(config)
321+
return base_config

‎deepctr/layers/interaction.py

+69-1
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
44
Authors:
55
Weichen Shen,[email protected],
6-
Harshit Pande
6+
Harshit Pande,
7+
Yi He, [email protected]
78
89
"""
910

@@ -26,6 +27,7 @@
2627

2728
from .activation import activation_layer
2829
from .utils import concat_func, reduce_sum, softmax, reduce_mean
30+
from .core import DNN
2931

3032

3133
class AFMLayer(Layer):
@@ -1489,3 +1491,69 @@ def get_config(self):
14891491
'regularizer': self.regularizer,
14901492
})
14911493
return config
1494+
1495+
1496+
class BridgeModule(Layer):
1497+
"""Bridge Module used in EDCN
1498+
1499+
Input shape
1500+
- A list of two 2D tensor with shape: ``(batch_size, units)``.
1501+
1502+
Output shape
1503+
- 2D tensor with shape: ``(batch_size, units)``.
1504+
1505+
Arguments
1506+
- **bridge_type**: The type of bridge interaction, one of 'pointwise_addition', 'hadamard_product', 'concatenation', 'attention_pooling'
1507+
1508+
- **activation**: Activation function to use.
1509+
1510+
References
1511+
- [Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models.](https://dlp-kdd.github.io/assets/pdf/DLP-KDD_2021_paper_12.pdf)
1512+
1513+
"""
1514+
1515+
def __init__(self, bridge_type='hadamard_product', activation='relu', **kwargs):
1516+
self.bridge_type = bridge_type
1517+
self.activation = activation
1518+
1519+
super(BridgeModule, self).__init__(**kwargs)
1520+
1521+
def build(self, input_shape):
1522+
if not isinstance(input_shape, list) or len(input_shape) < 2:
1523+
raise ValueError(
1524+
'A `BridgeModule` layer should be called '
1525+
'on a list of 2 inputs')
1526+
1527+
self.dnn_dim = int(input_shape[0][-1])
1528+
if self.bridge_type == "concatenation":
1529+
self.dense = Dense(self.dnn_dim, self.activation)
1530+
elif self.bridge_type == "attention_pooling":
1531+
self.dense_x = DNN([self.dnn_dim, self.dnn_dim], self.activation, output_activation='softmax')
1532+
self.dense_h = DNN([self.dnn_dim, self.dnn_dim], self.activation, output_activation='softmax')
1533+
1534+
super(BridgeModule, self).build(input_shape) # Be sure to call this somewhere!
1535+
1536+
def call(self, inputs, **kwargs):
1537+
x, h = inputs
1538+
if self.bridge_type == "pointwise_addition":
1539+
return x + h
1540+
elif self.bridge_type == "hadamard_product":
1541+
return x * h
1542+
elif self.bridge_type == "concatenation":
1543+
return self.dense(tf.concat([x, h], axis=-1))
1544+
elif self.bridge_type == "attention_pooling":
1545+
a_x = self.dense_x(x)
1546+
a_h = self.dense_h(h)
1547+
return a_x * x + a_h * h
1548+
1549+
def compute_output_shape(self, input_shape):
1550+
return (None, self.dnn_dim)
1551+
1552+
def get_config(self):
1553+
base_config = super(BridgeModule, self).get_config().copy()
1554+
config = {
1555+
'bridge_type': self.bridge_type,
1556+
'activation': self.activation
1557+
}
1558+
config.update(base_config)
1559+
return config

0 commit comments

Comments
 (0)