Skip to content

create files in structure to serialize to disk#24

Closed
maeg02 wants to merge 6 commits intoBiznamics:masterfrom
maeg02:master
Closed

create files in structure to serialize to disk#24
maeg02 wants to merge 6 commits intoBiznamics:masterfrom
maeg02:master

Conversation

@maeg02
Copy link

@maeg02 maeg02 commented Mar 8, 2020

DRAFT DRAFT DRAFT

With that said, I have created something that you can review (dismiss), and hopefully give me some pointers if this is the correct setup.

ShuffleRunner would create one of these ShufflerDataSplitter. (Or inject somehow)

This would take the data already exported, as ShuffleBlocks blocks, then recreate them as individual blocks and send them to the already existing shuffle.Serialize.

Could the be something to continue on?

@rappen
Copy link
Contributor

rappen commented Mar 10, 2020

Looks like it only splits, so it would be a one-way split for version control after export, but no way of combining them back for import?
I think if this gets implemented, there really should be a way to import from that format too, to complete the story.

@rappen
Copy link
Contributor

rappen commented Mar 20, 2020

Ping @maeg02 - any comment?

@rappen
Copy link
Contributor

rappen commented Mar 29, 2020

Fun Fact:
Early versions of the Shuffle exported all blocks as individual files, which were finally zipped together with the definition file to a single distributable file.
When importing you selected the zip file, which was unpacked to a temp folder and then imported.

I'm not sure it would be so great to go back to that pattern. If you really want individual files, you can create one definition for each block. On the other hand that would break the functionality build into the Relation element, and id mapping during import does not work across multiple files.

@maeg02
Copy link
Author

maeg02 commented Apr 19, 2020

Still missing merge function...
Would you like to recommend a place to put the split option?

@maeg02
Copy link
Author

maeg02 commented Apr 25, 2020

Now the first version of Merge is in there as well, could this work?

Pointers to consider:
1)
Passing around a Dictionary<string, XmlDocument>, is this a good choice, the first string is really two parts "<block name / Directory name>/<file name, record id>"
Should this string be its own class? If so where to put it, etc
Should this Dictionary be encapsulated to have methods on it like ToString or something?

I have removed Tuple in favour of "( )" return values, these can have names and that is better than Item1, Item2, Item3 etc.
Keep or remove?

What do you think?

@maeg02
Copy link
Author

maeg02 commented Jun 2, 2020

Poking this thread, to see if we can get it moving again

@rappen
Copy link
Contributor

rappen commented Jun 4, 2020

I'd love to have a look at this, but been busy for quite a while 😕

@maeg02
Copy link
Author

maeg02 commented Jul 2, 2020

Last try, any feedback?
Regards Markus

@maeg02 maeg02 closed this by deleting the head repository Jan 31, 2025
@rappen
Copy link
Contributor

rappen commented Jan 31, 2025

Oh... sorry... I dropped the ball.
Still relevant?

@maeg02
Copy link
Author

maeg02 commented Feb 3, 2025

No, not really, let us close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants