Skip to content

Commit a3ff6df

Browse files
kjytaykjytay
authored
Remove extraneous BSD licenses, minor code changes for consistency, minor wiki changes. (#31)
1 parent 8a838a8 commit a3ff6df

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+31
-1181
lines changed

‎build.gradle‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ allprojects {
2626

2727
group = "com.linkedin.dualip"
2828

29-
project.version = "2.7.1"
29+
project.version = "2.7.2"
3030

3131
repositories {
3232
mavenCentral()

‎docs/citing/index.rst‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Citing DuaLip
44
=============
55

6-
DuaLip has been created on the basis of the following research paper. If you cite DuaLip, please use the following:
6+
DuaLip has been created on the basis of the following research papers. To cite DuaLip, please use the following:
77

88

99
.. code:: latex
@@ -16,7 +16,6 @@ DuaLip has been created on the basis of the following research paper. If you cit
1616
url = {https://arxiv.org/abs/2103.05277}
1717
}
1818

19-
2019
@InProceedings{pmlr-v119-basu20a,
2120
title = {{ECLIPSE}: An Extreme-Scale Linear Program Solver for Web-Applications},
2221
author = {Basu, Kinjal and Ghoting, Amol and Mazumder, Rahul and Pan, Yao},

‎docs/interfaces/parameters.rst‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -10,24 +10,24 @@ The solver settings are displayed in the following table.
1010
===================================== ============= ============== ==============================================================================================================
1111
Parameters Is Required Default Value Description
1212
===================================== ============= ============== ==============================================================================================================
13-
:code:`driver.objectiveClass` true N/A Objective function implementation
14-
:code:`driver.solverOutputPath` true N/A The outputPath of solver
15-
:code:`driver.initialLambdaPath` false N/A Optional path to initialize dual variables for algorithm restarts
16-
:code:`driver.gamma` false 1E-3 Coefficient for quadratic objective regularizer, used by most objectives
17-
:code:`driver.outputFormat` false AVRO The format of output, can be AVRO, JSON or ORC
18-
:code:`driver.projectionType` true N/A One of available projections (simplex, simplexInequality, unitbox, boxcut, etc.)
19-
:code:`driver.boxCutUpperBound` false 1 Upper bound for the box cut projection constraint
20-
:code:`driver.savePrimal` false false Flag to save primal
21-
:code:`driver.verbosity` false 1 The levels of logging to be shown. We currently support 0 and 1.
22-
:code:`input.ACblocksPath` true N/A Path of matrix A & c encoded as data blocks
23-
:code:`input.vectorBPath` true N/A Path of vector b
24-
:code:`input.format` true N/A The format of input data, e.g. AVRO, JSON, CSV or ORC
25-
:code:`optimizer.maxIter` true N/A The maximum number of iterations the solver will run
26-
:code:`optimizer.solverType` true N/A The type of optimizer, currently supported: AGD, LBFGSB, and LBFGS
27-
:code:`optimizer.designInequality` false true True if Ax <= b, false if Ax = b or have mixed constraints
28-
:code:`optimizer.mixedDesignPivotNum` false 0 The pivot number if we have mixed A_1x <= b1 and A_2x = b2, i.e. how many inequality constraints come first
29-
:code:`optimizer.dualTolerance` true N/A Tolerance criteria for dual variable change
30-
:code:`optimizer.slackTolerance` true N/A Tolerance criteria for slack
13+
:code:`driver.initialLambdaPath` false N/A Filepath to initialize dual variables for algorithm restarts (optional).
14+
:code:`driver.gamma` false 1E-3 Coefficient for quadratic objective regularizer, used by most objectives.
15+
:code:`driver.projectionType` true N/A Type of projection used (simplex, simplexInequality, unitbox, boxcut, etc.).
16+
:code:`driver.boxCutUpperBound` false 1 Upper bound for the box cut projection constraint.
17+
:code:`driver.objectiveClass` true N/A Class name of the objective function, one of "MooSolverDualObjectiveFunction", "MatchingSolverDualObjectiveFunction", "ConstrainedMatchingSolverDualObjectiveFunction", "ParallelMooSolverDualObjectiveFunction".
18+
:code:`driver.outputFormat` false AVRO The format of output, can be AVRO, JSON or ORC.
19+
:code:`driver.savePrimal` false false Flag to save primal variable values at the solution.
20+
:code:`driver.verbosity` false 1 The levels of logging to be shown. We currently support 0 (concise logging) and 1 (log with increased verbosity).
21+
:code:`driver.solverOutputPath` true N/A Directory path to save solution at.
22+
:code:`input.ACblocksPath` true N/A Path of matrix A & c encoded as data blocks.
23+
:code:`input.vectorBPath` true N/A Path of vector b.
24+
:code:`input.format` true N/A The format of input data, e.g. AVRO, JSON, CSV or ORC.
25+
:code:`optimizer.maxIter` true N/A The maximum number of iterations the solver will run.
26+
:code:`optimizer.solverType` true N/A The type of optimizer, currently supported: AGD, LBFGSB, and LBFGS.
27+
:code:`optimizer.designInequality` false true True if Ax <= b, false if Ax = b or have mixed constraints.
28+
:code:`optimizer.mixedDesignPivotNum` false 0 The pivot number if we have mixed A_1x <= b1 and A_2x = b2, i.e. how many inequality constraints come first.
29+
:code:`optimizer.dualTolerance` true N/A Tolerance criteria for dual variable change.
30+
:code:`optimizer.slackTolerance` true N/A Tolerance criteria for slack.
3131
===================================== ============= ============== ==============================================================================================================
3232

3333

‎docs/solver/index.rst‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -79,9 +79,9 @@ The Algorithm
7979

8080
The overall algorithm can now be written as:
8181

82-
1. Start with an initial :math:`\lambda`
83-
2. Get Primal: :math:`x_{\gamma}^*(\lambda)`
84-
3. Get Gradient: :math:`Ax_{\gamma}^*(\lambda) - b`
82+
1. Start with an initial :math:`\lambda`.
83+
2. Get Primal: :math:`x_{\gamma}^*(\lambda)`.
84+
3. Get Gradient: :math:`Ax_{\gamma}^*(\lambda) - b`.
8585
4. Update :math:`\lambda` via appropriate mechanisms.
8686
5. Continue till converge.
8787

‎dualip/src/main/scala/com/linkedin/dualip/blas/VectorOperations.scala‎

Lines changed: 0 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,3 @@
1-
/*
2-
* BSD 2-CLAUSE LICENSE
3-
*
4-
* Copyright 2021 LinkedIn Corporation
5-
* All Rights Reserved.
6-
*
7-
* Redistribution and use in source and binary forms, with or without
8-
* modification, are permitted provided that the following conditions
9-
* are met:
10-
* 1. Redistributions of source code must retain the above copyright
11-
* notice, this list of conditions and the following disclaimer.
12-
* 2. Redistributions in binary form must reproduce the above copyright
13-
* notice, this list of conditions and the following disclaimer in the
14-
* documentation and/or other materials provided with the distribution.
15-
*
16-
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
17-
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18-
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19-
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
20-
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21-
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
22-
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
23-
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
24-
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
25-
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
26-
* SUCH DAMAGE.
27-
*/
28-
291
package com.linkedin.dualip.blas
302

313
import breeze.linalg.{SparseVector => BSV}

‎dualip/src/main/scala/com/linkedin/dualip/problem/MatchingSlateSolver.scala‎

Lines changed: 6 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,3 @@
1-
/*
2-
* BSD 2-CLAUSE LICENSE
3-
*
4-
* Copyright 2021 LinkedIn Corporation
5-
* All Rights Reserved.
6-
*
7-
* Redistribution and use in source and binary forms, with or without
8-
* modification, are permitted provided that the following conditions
9-
* are met:
10-
* 1. Redistributions of source code must retain the above copyright
11-
* notice, this list of conditions and the following disclaimer.
12-
* 2. Redistributions in binary form must reproduce the above copyright
13-
* notice, this list of conditions and the following disclaimer in the
14-
* documentation and/or other materials provided with the distribution.
15-
*
16-
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
17-
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18-
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19-
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
20-
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21-
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
22-
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
23-
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
24-
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
25-
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
26-
* SUCH DAMAGE.
27-
*/
28-
291
package com.linkedin.dualip.problem
302

313
import breeze.linalg.{SparseVector => BSV}
@@ -84,7 +56,7 @@ class MatchingSolverDualObjectiveFunction(
8456
override def getSardBound(lambda: BSV[Double]): Double = {
8557
val lambdaArray: Broadcast[Array[Double]] = spark.sparkContext.broadcast(lambda.toArray) // for performance
8658
val aggregator = new Tuple5Semigroup[Int, Int, Double, Max[Double], Max[Int]]
87-
val (nonVertexSoln, numI, corralSize, corralSizeMax, jMax) = problemDesign.map { block =>
59+
val (nonVertexSoln, numI, corralSize, corralSizeMax, jMax) = problemDesign.map { case block =>
8860
// below line is the same cost as a projection
8961
val (nV, corral, jM) = slateOptimizer.sardBound(block, lambdaArray.value)
9062
(nV, 1, corral, Max(corral), jM)
@@ -117,8 +89,9 @@ class MatchingSolverDualObjectiveFunction(
11789
*/
11890
def getPrimal(lambda: BSV[Double]): Dataset[(String, Seq[Slate])] = {
11991
val lambdaArray: Array[Double] = lambda.toArray // for performance
120-
problemDesign.map(block =>
121-
(block.id, slateOptimizer.optimize(block, lambdaArray)))
92+
problemDesign.map { case block =>
93+
(block.id, slateOptimizer.optimize(block, lambdaArray))
94+
}
12295
}
12396

12497
/**
@@ -207,7 +180,8 @@ object MatchingSolverDualObjectiveFunction extends DualPrimalObjectiveLoader {
207180
blocks = blocks.withColumn(field, lit(null))
208181
}
209182
}
210-
val data = blocks.as[DataBlock]
183+
val data = blocks.withColumnRenamed("memberId", "id")
184+
.as[DataBlock]
211185
.repartition(spark.sparkContext.defaultParallelism)
212186
.persist(StorageLevel.MEMORY_ONLY)
213187

‎dualip/src/main/scala/com/linkedin/dualip/problem/MooSolver.scala‎

Lines changed: 0 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,3 @@
1-
/*
2-
* BSD 2-CLAUSE LICENSE
3-
*
4-
* Copyright 2021 LinkedIn Corporation
5-
* All Rights Reserved.
6-
*
7-
* Redistribution and use in source and binary forms, with or without
8-
* modification, are permitted provided that the following conditions
9-
* are met:
10-
* 1. Redistributions of source code must retain the above copyright
11-
* notice, this list of conditions and the following disclaimer.
12-
* 2. Redistributions in binary form must reproduce the above copyright
13-
* notice, this list of conditions and the following disclaimer in the
14-
* documentation and/or other materials provided with the distribution.
15-
*
16-
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
17-
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18-
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19-
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
20-
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21-
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
22-
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
23-
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
24-
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
25-
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
26-
* SUCH DAMAGE.
27-
*/
28-
291
package com.linkedin.dualip.problem
302

313
import breeze.linalg.{SparseVector => BSV}

‎dualip/src/main/scala/com/linkedin/dualip/projection/BoxCutProjection.scala‎

Lines changed: 0 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,3 @@
1-
/*
2-
* BSD 2-CLAUSE LICENSE
3-
*
4-
* Copyright 2021 LinkedIn Corporation
5-
* All Rights Reserved.
6-
*
7-
* Redistribution and use in source and binary forms, with or without
8-
* modification, are permitted provided that the following conditions
9-
* are met:
10-
* 1. Redistributions of source code must retain the above copyright
11-
* notice, this list of conditions and the following disclaimer.
12-
* 2. Redistributions in binary form must reproduce the above copyright
13-
* notice, this list of conditions and the following disclaimer in the
14-
* documentation and/or other materials provided with the distribution.
15-
*
16-
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
17-
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18-
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19-
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
20-
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21-
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
22-
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
23-
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
24-
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
25-
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
26-
* SUCH DAMAGE.
27-
*/
28-
291
package com.linkedin.dualip.projection
302

313
import breeze.linalg.{SparseVector => BSV}

‎dualip/src/main/scala/com/linkedin/dualip/projection/GreedyProjection.scala‎

Lines changed: 0 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,3 @@
1-
/*
2-
* BSD 2-CLAUSE LICENSE
3-
*
4-
* Copyright 2021 LinkedIn Corporation
5-
* All Rights Reserved.
6-
*
7-
* Redistribution and use in source and binary forms, with or without
8-
* modification, are permitted provided that the following conditions
9-
* are met:
10-
* 1. Redistributions of source code must retain the above copyright
11-
* notice, this list of conditions and the following disclaimer.
12-
* 2. Redistributions in binary form must reproduce the above copyright
13-
* notice, this list of conditions and the following disclaimer in the
14-
* documentation and/or other materials provided with the distribution.
15-
*
16-
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
17-
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18-
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19-
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
20-
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21-
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
22-
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
23-
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
24-
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
25-
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
26-
* SUCH DAMAGE.
27-
*/
28-
291
package com.linkedin.dualip.projection
302

313
import breeze.linalg.{argmax, SparseVector => BSV}

‎dualip/src/main/scala/com/linkedin/dualip/projection/PolytopeProjection.scala‎

Lines changed: 0 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,3 @@
1-
/*
2-
* BSD 2-CLAUSE LICENSE
3-
*
4-
* Copyright 2021 LinkedIn Corporation
5-
* All Rights Reserved.
6-
*
7-
* Redistribution and use in source and binary forms, with or without
8-
* modification, are permitted provided that the following conditions
9-
* are met:
10-
* 1. Redistributions of source code must retain the above copyright
11-
* notice, this list of conditions and the following disclaimer.
12-
* 2. Redistributions in binary form must reproduce the above copyright
13-
* notice, this list of conditions and the following disclaimer in the
14-
* documentation and/or other materials provided with the distribution.
15-
*
16-
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
17-
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18-
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19-
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
20-
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21-
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
22-
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
23-
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
24-
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
25-
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
26-
* SUCH DAMAGE.
27-
*/
28-
291
package com.linkedin.dualip.projection
302

313
import breeze.linalg.{DenseMatrix => BDM, DenseVector => BDV, SparseVector => BSV}

0 commit comments

Comments
 (0)