Feature: set real source type in status

[kebe7jun]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @kebe7jun, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant enhancement to the Dataset resource's status by adding a sourceType field. The primary objective is to provide a more precise and actionable representation of a dataset's origin, especially for REFERENCE datasets, by resolving and displaying their true underlying source type. This change improves the observability and clarity of dataset types within the Kubernetes environment, making it easier for users to understand the nature of their data sources at a glance.

Highlights

• Enhanced Dataset Status: A new sourceType field has been added to the DatasetStatus struct to store the actual type of the dataset's source, providing a more accurate representation than just the spec.
• Reference Dataset Resolution: The controller now intelligently resolves the sourceType for REFERENCE datasets. Instead of simply showing 'REFERENCE', it fetches the referenced dataset's status to determine and display its true underlying source type (e.g., S3, HDFS).
• Improved kubectl Output: The kubectl get dataset command has been updated to display the sourceType from the status field, offering users immediate and accurate visibility into the true source type of their datasets directly in the command-line output.
• Controller Logic Update: A new reconciliation step, reconcileStatusType, has been integrated into the dataset controller's reconciliation loop. This function is responsible for computing and persisting the status.sourceType based on the dataset's specification and, for reference types, by querying the referenced dataset's status.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a sourceType field to the Dataset status, which is intended to show the actual source type, particularly for REFERENCE datasets. The API/CRD changes are correct. The controller logic for resolving the source type has a potential issue in handling chained and circular references. I've provided a detailed comment and a suggested fix for this issue. The other changes in the PR are well-implemented.