Experiment Migration Management Command

[stephherbers]

Description

resolves #1664
second part here: #1724

User Impact

when run, all assistant and llm experiments will be converted to pipeline experiments

Demo

n/a

Docs and Changelog

n/a until the tool for users to do on their own is implemented

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[SmittieC]

Looking good! This is exciting.

I presume this logic is going to be re-used for the DIY button that users can click?

[snopoke]

in this instance you could skip the rest of the queries and just fetch that one experiment right? Or maybe also check that it's a working version and also fetch the default version if there is one - but either way I think it should be handles differently because the rest of the queries and filters don't make sense if you're already filtering down to a single object.

[snopoke]

Looking good.

[snopoke]

You could just do the select_related in the query on line 60 and then you wouldn't have to re-fetch this here.

You could also avoid the extra count later by assigning experiment_count = 1

[snopoke]

It may be that loading all of these into memory at once could be an issue so it would be prudent to use an iterator:

Suggested change

	experiments_to_convert = Experiment.objects.filter(id__in=combined_ids).select_related(
	"team", "assistant", "llm_provider", "llm_provider_model"
	)
	experiments_to_convert_queryset = Experiment.objects.filter(id__in=combined_ids).select_related(
	"team", "assistant", "llm_provider", "llm_provider_model"
	)
	# do this here since you can't do a count on an iterator
	experiment_count = experiments_to_convert_queryset.count()
	experiments_to_convert = experiments_to_convert_queryset.iterator(20)