feat(flotilla): Use max sources config to determine files per partition (#4535)

## Changes Made

Allow the number of files per partition in swordfish to be configurable
by config. For now.

Ideally the scheduler or optimizer will be able to size them better

## Related Issues

<!-- Link to related GitHub issues, e.g., "Closes #123" -->

## Checklist

- [ ] Documented in API Docs (if applicable)
- [ ] Documented in User Guide (if applicable)
- [ ] If adding a new documentation page, doc is added to
`docs/mkdocs.yml` navigation
- [ ] Documentation builds and is formatted properly (tag @/ccmao1130
for docs review)

Author: colin-ho

src/daft-distributed/src/pipeline_node/scan_source.rs

@@ -66,8 +66,9 @@ impl ScanSourceNode {
             return Ok(());
         }
 
-        for scan_task in self.scan_tasks.iter() {
-            let task = self.make_source_tasks(vec![scan_task.clone()].into())?;
+        let max_sources_per_scan_task = self.config.max_sources_per_scan_task;
+        for scan_tasks in self.scan_tasks.chunks(max_sources_per_scan_task) {
+            let task = self.make_source_tasks(scan_tasks.to_vec().into())?;
             if result_tx
                 .send(PipelineOutput::Task(SubmittableTask::new(task)))
                 .await