Skip to content

Commit a899eb2

Browse files
authored
Merge pull request #81 from heinezen/spec-rework
Specification improvement
2 parents 2d6fba8 + 100ac9d commit a899eb2

File tree

9 files changed

+1672
-1174
lines changed

9 files changed

+1672
-1174
lines changed

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ Which is totally possible with **nyan**.
116116
Specification
117117
-------------
118118

119-
Read the [specification](doc/nyan.md).
119+
Read the [specification](doc/nyan_specification.md).
120120

121121

122122
Current State of the Project

doc/README.md

+89-3
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,90 @@
1-
nyan
2-
====
1+
# nyan
32

4-
[language specification](./nyan.md)
3+
## WTF?
4+
5+
**nyan** is a strongly typed hierarchical key-value database with patch
6+
functionality and inheritance.
7+
8+
9+
## Design idea
10+
11+
[openage](https://github.com/SFTtech/openage) requires a very complex data
12+
storage to represent the hierarchy of its objects. Research and technology
13+
affects numerous units, civilization bonuses, monk conversions and all that
14+
with the goal to be ultimatively moddable by the community:
15+
16+
Current data representation formats make this nearly impossible to
17+
accomplish. Readability problems or huge lexical overhead led us to
18+
design a language crafted for our needs.
19+
20+
Enter **nyan**, which is our approach to store data in a new way™.
21+
22+
23+
## Core Principles
24+
25+
* Human-readable language
26+
* More or less compact (readability > memory)
27+
* General-purpose data definition + database features
28+
* Changing data with patches at runtime
29+
* Moddability of defined data
30+
* Portable
31+
* Object-oriented
32+
* Typesafe
33+
34+
35+
## Srsly?
36+
37+
Let's create a new unit with a mod: a japanese tentacle monster.
38+
39+
``` python
40+
TentacleMonster(Unit):
41+
name = "Splortsch"
42+
hp = 2000
43+
44+
Creation<TownCenter>():
45+
creates += {TentacleMonster}
46+
47+
TentacleMod(Mod):
48+
name = "Add the allmighty tentacle monster to your holy army"
49+
patches = {Creation}
50+
```
51+
52+
Things like `Unit` and `Mod` are provided by the game engine,
53+
`TownCenter` is provided by the base game data.
54+
55+
When the engine activates the mod, your town center can create the new unit.
56+
57+
58+
## Why nyan?
59+
60+
* nyan allows easy modding
61+
* Data packs ship configuration data and game content as `.nyan` files
62+
* Modpacks can change and extend existing information easily, by applying data "patches"
63+
* Patches are applied whenever the `libnyan` user decides when or where to do so
64+
* nyan is typesafe
65+
* The type of a member is stored when declaring it
66+
* The only things nyan can do: Hierarchical data declaration and patches
67+
* No member type casts
68+
* Only allowed operators for a member type can be called
69+
* nyan is invented here™
70+
* we can change the specification to our needs whenever we want
71+
72+
## Specification
73+
74+
A full specification is provided [here](nyan_specification.md).
75+
76+
## Integration into a Game Engine
77+
78+
* Some `.nyan` files are shipped with the game engine
79+
* They describe things the engine is capable of, basically the mod API
80+
* That way, the engine can be sure that things exist
81+
* The engine can access all nyan file contents with type safety
82+
* The data files of the game then extend and change the API `nyan::Object`s
83+
84+
* The nyan database provides a C++ API used by the game engine
85+
* Can parse `.nyan` files and add all information to the database
86+
* Provides hooks so the engine can react on internal changes
87+
88+
* Data does not contain any executable code but can specify function names
89+
and parameters. The game engine is responsible for calling those
90+
functions or redirecting to custom scripts

doc/embedding.md

+207
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
# Embedding nyan into a Game Engine
2+
3+
## nyan interpreter
4+
5+
`.nyan` files are read by the nyan interpreter part of `libnyan`.
6+
7+
* You feed `.nyan` files into the `nyan::Database`
8+
* All data is loaded and checked for validity
9+
* You can query any member and object of the store
10+
* You can hold `nyan::Object`s as handles
11+
* You can apply patches to any object at a given time, all already-applied patches after that time are undone
12+
* All data history is stored over time
13+
14+
15+
## Embedding in the Engine Code
16+
17+
The mod API definitions in `engine.nyan` have to be designed exacly the way the
18+
C++ engine code is then using it. It sets up the type system so that the nyan
19+
C++ API can then be used to provide the correct information to the program that embeds nyan.
20+
21+
The load procedure and data access could be done like this:
22+
23+
1. Load `engine.nyan`
24+
1. Read `pack.nfo`
25+
1. Load `pack.nyan`
26+
1. Apply "mod-activating" patches in `pack.DefaultMod`
27+
1. Let user select one of `engine.StartConfigs.available`
28+
1. Generate a map and place the `CFG.initial_buildings`
29+
1. Display creatable units for each building on that map
30+
31+
When the newly created villager is selected, it can build towncenters!
32+
And the towncenter can research a healthpoint-upgrade for villagers.
33+
34+
``` cpp
35+
// callback function for reading nyan files via the engine
36+
// we need this so nyan can access into e.g. archives of the engine.
37+
std::string base_path = "/some/game/root";
38+
auto file_fetcher = [base_path] (const std::string &filename) {
39+
return std::make_shared<File>(base_path + '/' + filename);
40+
};
41+
42+
// initialization of API
43+
auto db = std::make_shared<nyan::Database>();
44+
db->load("engine.nyan", file_fetcher);
45+
46+
// load the userdata
47+
ModInfo nfo = read_mod_file("pack.nfo");
48+
db->load(nfo.load, file_fetcher);
49+
50+
// modification view: this is the changed database state
51+
std::shared_ptr<nyan::View> root = db->new_view();
52+
53+
nyan::Object mod_obj = root->get(nfo.mod);
54+
if (not mod_obj.extends("engine.Mod", 0)) { error(); }
55+
56+
nyan::OrderedSet mod_patches = mod_obj.get<nyan::OrderedSet>("patches", 0);
57+
58+
// activation of userdata (at t=0)
59+
nyan::Transaction mod_activation = root->new_transaction(0);
60+
61+
for (auto &patch : mod_patches.items<nyan::Patch>()) {
62+
mod_activation.add(patch);
63+
}
64+
65+
if (not mod_activation.commit()) { error("failed transaction"); }
66+
67+
// presentation of userdata (t=0)
68+
for (auto &obj : root->get("engine.StartConfigs").get<nyan::Set>("available", 0).items<nyan::Object>()) {
69+
present_in_selection(obj);
70+
}
71+
72+
// feedback from ui
73+
nyan::Object selected_startconfig = ...;
74+
75+
// use result of ui-selection
76+
printf("generate map with config %s", selected_startconfig.get<nyan::Text>("name", 0));
77+
place_buildings(selected_startconfig.get<nyan::Set>("initial_buildings", 0));
78+
79+
// set up teams and players
80+
auto player0 = std::make_shared<nyan::View>(root);
81+
auto player1 = std::make_shared<nyan::View>(root);
82+
83+
84+
// ====== let's assume the game runs now
85+
run_game();
86+
87+
88+
// to check if a unit is dead:
89+
engine::Unit engine_unit = ...;
90+
nyan::Object unit_type = engine_unit.get_type();
91+
int max_hp = unit_type.get<nyan::Int>("hp", current_game_time);
92+
float damage = engine_unit.current_damage();
93+
if (damage > max_hp) {
94+
engine_unit.die();
95+
}
96+
else {
97+
engine_unit.update_hp_bar(max_hp - damage);
98+
}
99+
100+
// to display what units a selected entity can build:
101+
nyan::Object selected = get_selected_object_type();
102+
if (selected.extends("engine.Unit", current_game_time)) {
103+
for (auto &unit : selected.get<nyan::Set>("can_create", current_game_time).items<nyan::Object>()) {
104+
display_creatable(unit);
105+
}
106+
}
107+
108+
// technology research:
109+
nyan::Object tech = get_tech_to_research();
110+
std::shared_ptr<nyan::View> &target = target_player();
111+
nyan::Transaction research = target.new_transaction(current_game_time);
112+
for (auto &patch : tech.get<nyan::Orderedset>("patches", current_game_time).items<nyan::Patch>()) {
113+
research.add(patch);
114+
}
115+
116+
if (not research.commit()) { error("failed transaction"); }
117+
```
118+
119+
120+
### Database views
121+
122+
Problem: Different players and teams have different states of the same nyan tree.
123+
124+
Solution: Hierarchy of state views.
125+
126+
A `nyan::View` has a parent which is either the root database or another `nyan::View`.
127+
128+
The view then stores the state for e.g. a player.
129+
130+
What does that mean?
131+
132+
* You can create a view of the main database
133+
* You can create a view of a view
134+
* Querying values respects the view the query is executed in
135+
* If a patch is applied in a view, the data changes are applied in this view
136+
and all children of it. Parent view remain unaffected.
137+
138+
Querying data works like this:
139+
* `nyan::Object obj = view.get(object_name)`
140+
* The `nyan::Object` is just a handle which is then used for real queries
141+
* `obj.get(member_name, time)` will evaluates the member of the object at a give time
142+
* This returns the `nyan::Value` stored in the member at the given time.
143+
144+
Patching data works as follows:
145+
* Obtain a patch object from some view
146+
* `nyan::Object patch = view.get(patch_name);`
147+
* If it is known in the view, return it
148+
* Else return it from the parent view
149+
* Create a transaction with this Patch to change the view state at the desired time
150+
* `nyan::Transaction tx = view.new_transaction(time);`
151+
* Add one or more patch objects to the transaction
152+
* `tx.add(patch); tx.add(...);`
153+
* `tx.add(another_patch, view.get(target_object_name))` is used to patch a child of
154+
the patch target.
155+
* Commit the transaction
156+
* `bool success = tx.commit();`
157+
* This triggers, for each patch in the transaction:
158+
* Determine the patch target object name
159+
* If a custom patch target was requested,
160+
check if it was a child of the default patch target at loadtime.
161+
* Copy the patch target object in a (new) state at `time`
162+
* Query the view of the transaction at `time` for the target object, this may recursively query parent views
163+
* If there is no state at `time` in the view of the transaction, create a new state
164+
* Copy the target object into the state at `time` in the view of the transaction
165+
* Linearize the inheritance hierary to a list of patch objects
166+
* e.g. if we have a `SomePatch<TargetObj>()` and `AnotherPatch(SomePatch)` and we would like to apply `AnotherPatch`, this will result in `[SomePatch, AnotherPatch]`
167+
* Apply the list left to right and modify the copied target object
168+
* Notify child views that this patch was applied, perform the patch there as well
169+
170+
This approach allows different views of the database state and integrates with the
171+
patch idea so e.g. team boni and player specific updates can be handled in an "easy"
172+
way.
173+
174+
175+
#### API definition example
176+
177+
openage uses an [ECS-style nyan API](https://github.com/SFTtech/openage/tree/master/doc/nyan/api_reference) for storing game data.
178+
179+
180+
### Creating a scripting API
181+
182+
nyan does provide any possibility to execute code.
183+
But nyan can be used as entry-point for full dynamic scripting APIs:
184+
The names of hook functions to be called are set up through nyan.
185+
The validity of code that is called that way is impossible to check,
186+
so this can lead to runtime crashes.
187+
188+
189+
## nyanc - the nyan compiler
190+
191+
**nyanc** can compile a .nyan file to a .h and .cpp file, this just creates
192+
a new nyan type the same way the primitive types from above are defined.
193+
194+
Members can then be acessed directly from C++.
195+
196+
The only problem still unsolved with `nyanc` is:
197+
198+
If a "non-optimized" `nyan::Object` has multiple parents where some of them
199+
were "optimized" and made into native code by `nyanc`, we can't select
200+
which of the C++ objects to instanciate for it. And we can't create the
201+
combined "optimized" object as the `nyan::Object` appeared at runtime.
202+
203+
This means we have to provide some kind of annotation, which of the parents
204+
should be the annotated ones.
205+
206+
Nevertheless, `nyanc` is just an optimization, and has therefore no
207+
priority until we need it.

doc/format_version

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
2

0 commit comments

Comments
 (0)