|
1 | | -# neo4j |
| 1 | +# json to cypher |
| 2 | +* graph data formats : comparing multiple representations formats of the same graph, cypher offers an impressively compact and comprehensible notation. Unfortunately, cypher is a domain specific language, therfore automated import and export requires a native json format that offers generic hierarchical maps of lists. |
| 3 | +Therefore, the goal is to provide utilities that can convers multiple json formats that are conetnt agnostic into cypher. |
| 4 | +* native data structures : Applications usually do know the relations between the data structures therefore tend to optimize the data export and do not embed explicit relations in the data structures, that results in a more packed data than if an explicit list of nodes and edges has to be created. |
| 5 | + |
| 6 | + |
| 7 | + |
| 8 | +* jsonl : json lines is very effitient when it comes to scale as every line can be processed separately without loasing the whole file in memory, such scale is outside of scope of this repo that targets applications that fit and can be processed in memory. |
| 9 | +* neo4j ids : althoug neo4j db has a unique id among all Lebel types, within the export, every Label has its own id scope, therefore when referencing a target within edges, the label has to be provided as well. |
| 10 | +* no ids : json data exchanged between applications should not contain app specific number ids because the import or append in an existing database will require assignment of new ids. Additionally, data created manually by users or other apps requires an additional effort to add those ids. |
| 11 | +* unique ids : The scope of these loading utilities is to deal with data where all members of every group can be uniquely identified with one of their properties that is human comprehensible such as name. If such a restriction is not possible, then a custom cypher will have to be written and automated json load cannot be used without given uids. Note that the name is used by neo4j as a Label (Text visualization) neo4j reserves the Label for the type and uses name instead. |
| 12 | +* uid maps, no lists : every group of data should be structured in a map, not a list, where the uid is used as a key |
| 13 | +* Labels are the terms used by neo4j for node types. inported data will not take advantage of multi typing capabilities of neo4j. Fallback on a single type allows to pack all data of the same type in a map where the key is the type of the nodes |
| 14 | + |
| 15 | +examples of `Packed json` : see `data\packed.json` and `data\zigbee_packed.json` |
| 16 | +```json |
| 17 | +{ |
| 18 | + "Persons":{ |
| 19 | + "Michael":{ |
| 20 | + "age": 41, |
| 21 | + "Persons":[ |
| 22 | + "Selina", |
| 23 | + "Rana", |
| 24 | + "Selma" |
| 25 | + ] |
| 26 | + }, |
| 27 | + "Selina":{}, |
| 28 | + "Rana":{}, |
| 29 | + "Selma":{} |
| 30 | + } |
| 31 | +} |
| 32 | +``` |
| 33 | + |
| 34 | +examples of `Mapped relations json` : see `data\persons_mapped_relations.json` and `data\programmers_mapped_relations.json` |
| 35 | + |
| 36 | +```json |
| 37 | +{ |
| 38 | + "Persons":{ |
| 39 | + "Michael":{ |
| 40 | + "age": 41, |
| 41 | + "knows":{ |
| 42 | + "Persons":[ |
| 43 | + "Selina", |
| 44 | + "Rana", |
| 45 | + "Selma" |
| 46 | + ], |
| 47 | + }, |
| 48 | + "children":{ |
| 49 | + "Persons":[ |
| 50 | + "Rana", |
| 51 | + "Selma" |
| 52 | + ] |
| 53 | + } |
| 54 | + }, |
| 55 | + "Selina":{}, |
| 56 | + "Rana":{}, |
| 57 | + "Selma":{} |
| 58 | + } |
| 59 | +} |
| 60 | +``` |
| 61 | + |
| 62 | +## Packed vs mapped relations |
| 63 | +* a `packed json` has minimal footprint and prevents redundancies as names of relations are usually not needed or known by the application using the data |
| 64 | +* a `mapped relations` is fully informative with regards to the relationship type, as it adds an extra step of grouping all relationship tapes in a map with the relationship Label as the key |
| 65 | +* a `packed json` can hava an optional file separate from the main data that Labels the relationships depending on the source and target |
| 66 | + |
| 67 | +example relations labels see also `data\zigbee_optional_relations.json` |
| 68 | +```json |
| 69 | +{ |
| 70 | + "Persons":{ |
| 71 | + "Persons":"knows" |
| 72 | + } |
| 73 | +} |
| 74 | +``` |
| 75 | +* only a `mapped relations` allows to have different relationship types between the same node types |
| 76 | +* a combination of both `packed json` and `mapped relations` is possible with automated identification depending on the property type (list, map) and its availability in the top node Types map. |
| 77 | +* both `packed json` and `mapped relations` can have target relation items as either a list of a map |
| 78 | + * a list a simpler in case the relation does not have any property |
| 79 | + * a map is needed if relatioships do have properties. The key is then the target and the value is a map of properties (see `data\persons_mapped_relations.json` with mapped relations weight) |
| 80 | + |
| 81 | +# neo4j usage |
2 | 82 | * Cypher : Graph Query Language |
3 | 83 | * apoc : Awesome Procedures On Cypher |
4 | 84 | * gds : Graph Data Science |
5 | | -## note |
6 | | -Please refer to the official links for latest documentation, this repo is only a sort of cheat sheet and contains history of commands I tested |
| 85 | + |
7 | 86 | # run |
8 | 87 | - docker: simply run `./run.sh` which content is |
9 | 88 | ```shell |
|
0 commit comments