Skip to content

Commit a1aa5af

Browse files
DogacelDogacel
committed
editions / proto2 support
1 parent 9299453 commit a1aa5af

File tree

67 files changed

+4025
-532
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

67 files changed

+4025
-532
lines changed

‎README.md‎

Lines changed: 144 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -11,21 +11,156 @@
1111
Generate kotlin data classes from `protobuf` files that supports _Kotlin Native_ that can be serialized and
1212
deserialized to protobuf using [kotlinx.serialization](https://github.com/Kotlin/kotlinx.serialization).
1313

14-
> [!NOTE]
15-
> Setup instructions and detailed documentation can be found
16-
> at [Setup and Documentation](https://dogacel.github.io/kotlinx-protobuf-gen).
17-
18-
![Demonstrate Code](./docs/src/doc/docs/assets/demonstrate_code.png)
14+
![Demonstrate Code](./assets/demonstrate_code.png)
1915

2016
## Features
2117

22-
- [x] Supports `proto2` and `proto3`.
18+
- [x] Supports `proto2`, `proto3` and `editions` up to `2023`.
2319
- [x] Generates `kotlinx.serialization` annotations for proto field numbers and serialization format.
2420
- [x] Generates Kotlin code for primitive fields such as `int32`, `string`, `bytes`.
2521
- [x] Generates Kotlin code for `message`, `enum`, `repeated`, `map`, `oneof` types.
2622
- [x] Generates Kotlin code that includes imports and uses nested types.
2723
- [x] Supports common well-known types such as `Timestamp`, `StringValue` and serializes them to kotlin
2824
primitives or standards.
25+
- [x] Support Well-Known Types deserialization to Well-Known Kotlin types such as `google.protobuf.Duration`
26+
to `kotlin.time.Duration` and `google.protobuf.Timestamp` to `kotlinx.datetime.Instant`.
27+
- An option is added to code generator to enable this feature.
28+
- More WKT additions will be added.
29+
30+
## Setup
31+
32+
### 1. Dependencies,
33+
34+
```kotlin
35+
plugins {
36+
kotlin("jvm") version "1.9.0"
37+
id("org.jetbrains.kotlin.plugin.serialization") version "1.9.0"
38+
id("com.google.protobuf") version "0.9.4"
39+
}
40+
41+
var protobufVersion = "3.23.4"
42+
43+
dependencies {
44+
// Runtime libraries include WKT conversion utilities and other libraries that are required such
45+
// as kotlinx.datetime, kotlinx.coroutines, kotlinx.serialization, etc.
46+
implementation("io.github.dogacel:kotlinx-protobuf-gen:0.0.1")
47+
}
48+
```
49+
50+
#### 2. Code generation,
51+
52+
```kotlin
53+
protobuf {
54+
protoc {
55+
artifact = "com.google.protobuf:protoc:$protobufVersion"
56+
}
57+
58+
plugins {
59+
id("kotlinx-protobuf-gen") {
60+
artifact = "io.github.dogacel:kotlinx-protobuf-gen:0.0.1:jvm8@jar"
61+
}
62+
}
63+
64+
// Enable Kotlin generation
65+
generateProtoTasks {
66+
all().forEach {
67+
it.builtins {
68+
remove("java") // Optionally you can keep the java generated files.
69+
}
70+
it.plugins {
71+
id("kotlinx-protobuf-gen") {
72+
option("package_prefix=custom.pkg") // Set a custom package prefix
73+
}
74+
}
75+
}
76+
}
77+
}
78+
```
79+
80+
#### 3. Writing proto files,
81+
82+
Add your proto files to a known proto file path such as `src/main/proto`.
83+
84+
```protobuf
85+
syntax = "proto3";
86+
87+
package demo;
88+
89+
message Task {
90+
int32 id = 1;
91+
optional string description = 2;
92+
Status status = 3;
93+
94+
enum Status {
95+
WIP = 0;
96+
DONE = 1;
97+
}
98+
}
99+
```
100+
101+
The following class will be generated and added to your classpath.
102+
103+
```kotlin
104+
@Serializable
105+
public data class Task(
106+
@ProtoNumber(number = 1)
107+
public val id: Int = 0,
108+
@ProtoNumber(number = 2)
109+
public val description: String? = null,
110+
@ProtoNumber(number = 3)
111+
public val status: Status = testgen.demo.Task.Status.WIP,
112+
) {
113+
@Serializable
114+
public enum class Status {
115+
@ProtoNumber(number = 0)
116+
WIP,
117+
118+
@ProtoNumber(number = 1)
119+
DONE,
120+
}
121+
}
122+
```
123+
124+
## Customizations
125+
126+
To customize the code generated, you can pass command line arguments or gradle options. For example,
127+
128+
```kotlin
129+
protobuf {
130+
protoc {
131+
artifact = "com.google.protobuf:protoc:$protobufVersion"
132+
}
133+
134+
plugins {
135+
id("kotlinx-protobuf-gen") {
136+
artifact = "io.github.dogacel:kotlinx-protobuf-gen:0.0.1:jvm8@jar"
137+
}
138+
}
139+
140+
// Enable Kotlin generation
141+
generateProtoTasks {
142+
all().forEach {
143+
it.builtins {
144+
remove("java") // Optionally you can keep the java generated files.
145+
}
146+
it.plugins {
147+
id("kotlinx-protobuf-gen") {
148+
option("package_prefix=custom.pkg") // Set a custom package prefix
149+
}
150+
}
151+
}
152+
}
153+
}
154+
```
155+
156+
### Available Options
157+
158+
| Option | Description | Default |
159+
|--------------------|------------------------------------------------------------------------------------------------------------|---------|
160+
| `package_prefix` | Prefix for the generated package names. Appended to the start of each class | `""` |
161+
| `useCamelCase` | Whether to use the original `snake_case` for proto fields or `camelCase`. Can be either `true` or `false`. | `true` |
162+
| `generateServices` | Whether to generate abstract gRPC stubs or not. Can be either `true` or `false`. | `true` |
163+
29164

30165
## Roadmap
31166

@@ -34,15 +169,8 @@ is a list of features we are working on that are required to release first stabl
34169

35170
- [ ] Proper serialization / deserialization of all types. Check "Known Issues" section below to see all major
36171
issues.
37-
- [ ] Run full conformance tests on the generated code.]
172+
- [ ] Run full conformance tests on the generated code.
38173
- [ ] Support Protobuf JSON format.
39-
40-
And here is a list of features that we are planning to work on after the first stable release.
41-
42-
- [x] Support Well-Known Types deserialization to Well-Known Kotlin types such as `google.protobuf.Duration`
43-
to `kotlin.time.Duration` and `google.protobuf.Timestamp` to `kotlinx.datetime.Instant`.
44-
- An option is added to code generator to enable this feature.
45-
- More WKT additions will be added.
46174
- [ ] Support various options such as `deprecated`, `default`, `json_name`.
47175
- [ ] Auto-generated comments from `.proto` files in the generated code.
48176
- [x] gRPC support.
@@ -51,6 +179,8 @@ And here is a list of features that we are planning to work on after the first s
51179
- [ ] Plugin and more option support for customizing the generated code. (Such as non-enforced nullability to
52180
gimmick proto2 required fields based on certain rules)
53181

182+
For the full list, check [issues](./issues)
183+
54184
## Known Issues
55185

56186
An issue to track `kotlinx.serialization`: https://github.com/Kotlin/kotlinx.serialization/issues/2401

‎app/build.gradle.kts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -99,11 +99,11 @@ val sourcesJar by tasks.registering(Jar::class) {
9999
from(kotlin.sourceSets.main.get().kotlin)
100100
}
101101

102-
val javadocJar by tasks.creating(Jar::class) {
102+
val javadocJar by tasks.registering(Jar::class) {
103103
group = JavaBasePlugin.DOCUMENTATION_GROUP
104104
description = "Assembles Javadoc JAR"
105105
archiveClassifier.set("javadoc")
106-
from(tasks.named("dokkaHtml"))
106+
from(tasks.dokkaGeneratePublicationHtml.flatMap { it.outputDirectory })
107107
}
108108

109109
publishing {
@@ -131,7 +131,7 @@ publishing {
131131
developer {
132132
id.set("dogacel")
133133
name.set("Doğaç Eldenk")
134-
email.set("dogacel@gmail.copm")
134+
email.set("dogacel@gmail.com")
135135
}
136136
}
137137
scm {

‎app/src/main/kotlin/dogacel/kotlinx/protobuf/gen/App.kt‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,14 @@ fun main() {
99

1010
val responseBuilder =
1111
PluginProtos.CodeGeneratorResponse.newBuilder()
12-
.setSupportedFeatures(PluginProtos.CodeGeneratorResponse.Feature.FEATURE_PROTO3_OPTIONAL_VALUE.toLong())
12+
.setSupportedFeatures(
13+
// Supported features is a bitwise OR of the following values:
14+
(
15+
PluginProtos.CodeGeneratorResponse.Feature.FEATURE_PROTO3_OPTIONAL_VALUE or
16+
PluginProtos.CodeGeneratorResponse.Feature.FEATURE_SUPPORTS_EDITIONS_VALUE
17+
).toLong(),
18+
)
19+
.setMaximumEdition(2023)
1320
.addAllFile(
1421
specs.map { spec ->
1522
PluginProtos.CodeGeneratorResponse.File.newBuilder()

‎app/src/main/kotlin/dogacel/kotlinx/protobuf/gen/CodeGenerator.kt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,7 @@ class CodeGenerator {
269269
// A trick to handle oneof fields. We need to make sure that only one of the fields is set.
270270
// Validation is done in `init` block so objects in invalid states can't be initialized.
271271
messageDescriptor.oneofs.forEach { oneOfDescriptor ->
272-
if (!oneOfDescriptor.isSynthetic && oneOfDescriptor.fields.isNotEmpty()) {
272+
if (oneOfDescriptor.fields.isNotEmpty()) {
273273
val codeSpec = CodeBlock.builder()
274274
codeSpec.addStatement("require(")
275275
codeSpec.indent()

‎app/src/main/kotlin/dogacel/kotlinx/protobuf/gen/DefaultValues.kt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ object DefaultValues {
2828
return CodeBlock.of("emptyList()")
2929
}
3030

31-
if (fieldDescriptor.hasOptionalKeyword()) {
31+
if (fieldDescriptor.hasPresence()) {
3232
return null
3333
}
3434

‎app/src/main/kotlin/dogacel/kotlinx/protobuf/gen/TypeNames.kt‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
package dogacel.kotlinx.protobuf.gen
22

33
import com.google.protobuf.Descriptors
4-
import com.squareup.kotlinpoet.ANY
54
import com.squareup.kotlinpoet.BOOLEAN
65
import com.squareup.kotlinpoet.BYTE_ARRAY
76
import com.squareup.kotlinpoet.ClassName
@@ -58,7 +57,10 @@ object TypeNames {
5857
Descriptors.FieldDescriptor.Type.BYTES -> BYTE_ARRAY
5958
Descriptors.FieldDescriptor.Type.STRING -> STRING
6059

61-
Descriptors.FieldDescriptor.Type.GROUP -> ANY
60+
Descriptors.FieldDescriptor.Type.GROUP ->
61+
typeNames[fieldDescriptor.messageType]
62+
?: throw IllegalStateException("Group type not found: ${fieldDescriptor.messageType.fullName}")
63+
6264
Descriptors.FieldDescriptor.Type.ENUM ->
6365
typeNames[fieldDescriptor.enumType]
6466
?: throw IllegalStateException("Enum type not found: ${fieldDescriptor.enumType.fullName}")
@@ -71,7 +73,7 @@ object TypeNames {
7173
}
7274

7375
if (
74-
fieldDescriptor.hasOptionalKeyword() ||
76+
fieldDescriptor.hasPresence() ||
7577
fieldDescriptor.type == Descriptors.FieldDescriptor.Type.MESSAGE ||
7678
fieldDescriptor.realContainingOneof != null
7779
) {

‎app/src/test/kotlin/dogacel/kotlinx/protobuf/gen/CodeGeneratorTest.kt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
package dogacel.kotlinx.protobuf.gen
22

3-
import messages.MessageNoFieldsOuterClass.MessageNoFields
3+
import messages.proto3.MessageNoFieldsOuterClass.MessageNoFields
44
import kotlin.test.Test
55
import kotlin.test.assertEquals
66

docs/src/doc/docs/assets/demonstrate_code.png renamed to assets/demonstrate_code.png

File renamed without changes.

‎build.gradle.kts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ plugins {
55
// Publishing
66
idea
77
`maven-publish`
8-
id("io.github.gradle-nexus.publish-plugin") version "2.0.0"
8+
alias(libs.plugins.io.github.gradle.nexus.publish.plugin)
99
}
1010

1111
group = "io.github.dogacel"
12-
version = "0.0.1"
12+
version = "0.1.0"
1313

1414
nexusPublishing {
1515
this.repositories {

‎docs/build.gradle.kts‎

Lines changed: 0 additions & 3 deletions
This file was deleted.

0 commit comments

Comments
 (0)