Do not require extra copy of dimensional goals aggregated data when evaluating without dimensions

[ondraz]

E.g. when evaluating aggregated data of test-mutli-dimension experiment in test_multi_dimension, we require to have another copy of aggregated data without dimensional columns see here.

It would be nice just to "group by" dimensional data without the need to have extra aggregated data without dimensions in agg goals dataframe.

current data:

test-multi-dimension		a	test_unit_type	global	exposure			1000	1000	1000	1000	1000
test-multi-dimension		b	test_unit_type	global	exposure			1001	1001	1001	1001	1001
test-multi-dimension		a	test_unit_type	unit	view	button-1	p-1	200	200	200	200	200
test-multi-dimension		b	test_unit_type	unit	view	button-1	p-1	220	220	220	220	220
test-multi-dimension		a	test_unit_type	unit	view	button-1		100	100	100	100	100
test-multi-dimension		b	test_unit_type	unit	view	button-1		180	180	180	180	180
test-multi-dimension		a	test_unit_type	unit	view			300	300	300	300	300
test-multi-dimension		b	test_unit_type	unit	view			400	400	400	400	400

data format requested in this issue:

test-multi-dimension		a	test_unit_type	global	exposure			1000	1000	1000	1000	1000
test-multi-dimension		b	test_unit_type	global	exposure			1001	1001	1001	1001	1001
test-multi-dimension		a	test_unit_type	unit	view	button-1	p-1	200	200	200	200	200
test-multi-dimension		b	test_unit_type	unit	view	button-1	p-1	220	220	220	220	220
test-multi-dimension		a	test_unit_type	unit	view	button-1		100	100	100	100	100
test-multi-dimension		b	test_unit_type	unit	view	button-1		180	180	180	180	180

[jancervenka]

@ondraz Hi Ondro, I think doing this might be tricky because the dataframe would then need to contain all possible combinations of element and product dimension values for the view goal (for example rows for button-2 and p-2. Otherwise, the group by would produce aggregations with missing data.

[ondraz]

If we just aggregate (sum) these 4 lines of agg. goal data,

test-multi-dimension		a	test_unit_type	unit	view	button-1	p-1	200	200	200	200	200
test-multi-dimension		b	test_unit_type	unit	view	button-1	p-1	220	220	220	220	220
test-multi-dimension		a	test_unit_type	unit	view	button-1		100	100	100	100	100
test-multi-dimension		b	test_unit_type	unit	view	button-1		180	180	180	180	180

we get exactly what we already have in the extra two lines with no dim values:

test-multi-dimension		a	test_unit_type	unit	view			300	300	300	300	300
test-multi-dimension		b	test_unit_type	unit	view			400	400	400	400	400

so:

1. goal count(test_unit_type.unit.view) - we can use four lines above and just sum them
2. goal count(test_unit_type.unit.view(element=button-1) - we filter four lines above by dim value and sum values

There's probably some argument we did it this way where we require those extra two lines with empty dim data but I don't recall it.

[jancervenka]

You're right that it works in this case but I don't think it would work in general.

1. Let's say there are 200 views with element = button-2 in the data that the DAO is selecting from.
2. But because there is no goal count(test_unit_type.unit.view(element=button-2)) in the experiment metrics, the button-2 views will not show up in the data frame.
3. Summing the dataframe rows with element = button-1 to produce count(test_unit_type.unit.view) will get us incorrect results because it will be missing the 200 button-2 views.

[jancervenka]

Also looking at the test-multi-dimension data, they kind of don't make sense. 😄

test-multi-dimension		a	test_unit_type	unit	view	button-1	p-1	200	200	200	200	200
test-multi-dimension		b	test_unit_type	unit	view	button-1	p-1	220	220	220	220	220
test-multi-dimension		a	test_unit_type	unit	view	button-1		100	100	100	100	100
test-multi-dimension		b	test_unit_type	unit	view	button-1		180	180	

For example, the third row contains all button-1 views from all products so its total count shouldn't be lower than the total count from the first row which represent button-1 views from p-1 product only. The first row views should be subset of the third row views.

I was just testing that the goal selection works correctly and didn't think about the specific values.