Skip to content

Conversation

@KCSAbeywickrama
Copy link
Contributor

@KCSAbeywickrama KCSAbeywickrama commented Nov 13, 2025

Purpose

Resolve wso2/product-ballerina-integrator#1757

Summary by CodeRabbit

Release Notes

  • New Features

    • GROUP_BY clause support in the query builder with automatic name generation for grouping keys.
    • Async unique-name generator exposed to the data mapper editor and clause UI.
    • New mapping options: sequence→primitive and sequence→array, plus improved sequence-type display and mapping flows.
  • Bug Fixes

    • Clause initialization and normalization improved for JOIN and GROUP_BY.
    • Focus/sequence metadata now preserved through mapping and IO representations.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 13, 2025

Walkthrough

Adds GROUP_BY intermediate clause support and sequence-mapping variants, propagates grouping metadata (groupById, isSeq, isFocused) through IO/DMModel and RPC utils, threads an async genUniqueName helper through UI/context, and updates mapping/menu logic to handle sequence-to-primitive/array mappings.

Changes

Cohort / File(s) Summary
Core types
workspaces/ballerina/ballerina-core/src/interfaces/data-mapper.ts
Added GROUP_BY = "group-by" to IntermediateClauseType, changed ORDER_BY value to "order-by", added isSeq?: boolean to IOType/IOTypeField, added groupById?: string to DMModel, and changed Query.fromClause type to IntermediateClause.
RPC / model building
workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
Propagate groupById and add isFocused/isSeq metadata when building IO types; temporarily update/restore model.groupById during array processing; import IntermediateClauseType.
View / unique-name helper
workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
Added genUniqueName helper (uses completions via BIDiagram RPC + TriggerKind), improved clause position handling, and passed genUniqueName into DataMapper component; replaced numeric triggerKind with TriggerKind enum.
Editor & context plumbing
workspaces/ballerina/data-mapper/src/index.tsx, workspaces/ballerina/data-mapper/src/components/DataMapper/DataMapperEditor.tsx, workspaces/ballerina/data-mapper/src/utils/DataMapperContext/DataMapperContext.ts
Added public prop genUniqueName to DataMapperEditorProps/exports, threaded genUniqueName into DataMapperContext constructor and public API, and added addClauses/genUniqueName signatures to IDataMapperContext.
Clause UI & defaults
workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClauseEditor.tsx, .../ClausesPanel.tsx
Added GROUP_BY option and localized labels; clause form uses "Grouping Key" label/docs for GROUP_BY; removed JOIN special-case in submit; ClausesPanel gains genUniqueName prop and fillDefaults which auto-initializes JOIN and GROUP_BY properties (GROUP_BY uses genUniqueName).
Mapping type variants
workspaces/ballerina/data-mapper/src/components/Diagram/Link/DataMapperLink/DataMapperLink.ts
Added SeqToPrimitive = "seq-primitive" and SeqToArray = "seq-array" to MappingType enum.
Diagram mapping logic
workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts, .../type-utils.ts, .../modification-utils.ts
getMappingType extended to choose SeqToPrimitive/SeqToArray for sequence sources; isPendingMappingRequired treats SeqToArray as not pending; getTypeName prefixes sequence types with "..." when isSeq; added mapSeqToX and mapSeqToArray utilities (LET clause creation, unique-name generation, GROUP_BY-aware placement).
Menu & mapping UI
workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx, workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts
Refactored menu generation with useMemo and unified progress wrapper; added sequence mapping menu entries and mapSeqToX usage; createLinkModel early-returns for SeqToArray after invoking mapSeqToX.
Diagram model integration
workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
Added queryClauses derived from model.query.intermediateClauses (LET/GROUP_BY) and included it in the query refetch key so diagram depends on intermediate clauses.

Sequence Diagram(s)

sequenceDiagram
    participant UI as Clause Editor UI
    participant Panel as ClausesPanel
    participant Context as DataMapperContext
    participant Model as DMModel

    UI->>Panel: User adds GROUP_BY clause (expr)
    Panel->>Panel: fillDefaults(clause) — set type="var"
    Panel->>Context: await genUniqueName(key, targetField)
    Context-->>Panel: uniqueName
    Panel->>Panel: clause.properties.name = uniqueName
    Panel->>Context: await addClauses(clause, targetField, isNew, index)
    Context->>Model: insert intermediateClause (GROUP_BY)
    Model->>Model: set groupById and propagate isSeq flags
Loading
sequenceDiagram
    participant Diagram as Diagram Model
    participant Utils as common-utils
    participant Mod as modification-utils
    participant Context as DataMapperContext

    Diagram->>Utils: determine mapping type for seq source
    Utils-->>Diagram: SeqToArray or SeqToPrimitive
    Diagram->>Mod: invoke mapSeqToX(link, context, modifier)
    Mod->>Context: genUniqueName(base, viewId)
    Context-->>Mod: uniqueName
    Mod->>Mod: create LET clause, position near GROUP_BY if present
    Mod->>Mod: update source field FQN refs
    Mod->>Diagram: createNewMapping(link, modifier)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

  • Focus areas for review:
    • MappingOptionsWidget.tsx — heavy refactor; verify memo deps, progress wrapper and conditional menu items.
    • modification-utils.ts — mapSeqToX LET creation, unique-name usage, and GROUP_BY-relative placement.
    • common-utils.ts / type-utils.ts — sequence detection heuristics and type-name formatting changes.
    • ClausesPanel.tsx / ClauseEditor.tsx — async fillDefaults with genUniqueName and clause property initialization.
    • DataMapperContext API additions — ensure callers supply new addClauses/genUniqueName and thread correctness.

Possibly related PRs

Suggested reviewers

  • hevayo
  • gigara
  • madushajg

Poem

🐰 A rabbit's ode to grouping found,
GROUP_BY seeds scattered all around.
Sequences fold to arrays or single prizes,
Names made unique in context rises,
Mapping hops forward—bright surprises! ✨

Pre-merge checks and finishing touches

❌ Failed checks (3 warnings)
Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is incomplete and lacks most required template sections. Only Purpose is provided, while Goals, Approach, UI Component Development checklist, Release notes, and other critical sections are missing. Complete the PR description by filling out all required sections from the template, particularly Goals, Approach, UI/UX considerations, release notes, and security checks.
Out of Scope Changes check ⚠️ Warning Changes include additional scope beyond the group by clause feature: new sequence mapping types (SeqToPrimitive, SeqToArray), sequence-to-primitive/array mapping logic, and refactored menu generation in MappingOptionsWidget that extend beyond the stated grouping requirement. Clarify whether sequence mapping enhancements are intentional additions or separate concerns; consider splitting sequence-related changes into a separate PR if unrelated to group by clause support.
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: adding group by clause support to the Data Mapper frontend UI, which aligns with the changeset's primary objective.
Linked Issues check ✅ Passed The PR successfully implements group by clause support across multiple layers: interface definitions (IntermediateClauseType.GROUP_BY), utility processing logic, UI components (ClauseEditor, ClausesPanel), and diagram handling, comprehensively addressing issue #1757's requirement.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@KCSAbeywickrama KCSAbeywickrama marked this pull request as ready for review November 26, 2025 10:52
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx (1)

141-151: Consider adding classes to the dependency array.

getItemElement references classes.itemContainer, but classes is not included in the useMemo dependency array. While Emotion's css() typically returns stable class name strings, it's a best practice to include all referenced values in the dependency array for correctness.

-    }, [pendingMappingType, link, context]);
+    }, [pendingMappingType, link, context, classes]);

Also applies to: 253-253

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4898189 and df42e40.

📒 Files selected for processing (15)
  • workspaces/ballerina/ballerina-core/src/interfaces/data-mapper.ts (5 hunks)
  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts (4 hunks)
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx (6 hunks)
  • workspaces/ballerina/data-mapper/src/components/DataMapper/DataMapperEditor.tsx (3 hunks)
  • workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClauseEditor.tsx (3 hunks)
  • workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx (3 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx (2 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/Link/DataMapperLink/DataMapperLink.ts (1 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts (2 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts (3 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts (2 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts (1 hunks)
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/type-utils.ts (1 hunks)
  • workspaces/ballerina/data-mapper/src/index.tsx (1 hunks)
  • workspaces/ballerina/data-mapper/src/utils/DataMapperContext/DataMapperContext.ts (3 hunks)
🧰 Additional context used
🧠 Learnings (25)
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/**/*.{ts,tsx} : All async operations and promise handling in diagram utilities should use async/await syntax instead of .then() callbacks

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts
📚 Learning: 2025-11-26T06:34:09.752Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:128-134
Timestamp: 2025-11-26T06:34:09.752Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, the goPrevViewBreadcrumb() method is only called when in a focused view, ensuring breadcrumbs are always present. No guard for empty breadcrumbs is needed.

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts
📚 Learning: 2025-11-26T07:49:56.428Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:136-141
Timestamp: 2025-11-26T07:49:56.428Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, the goPrevViewBackButton() method is only called when in a focused view, ensuring breadcrumbs are always present. No guard for empty breadcrumbs is needed.

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/Diagram.tsx : Implement input validation for CDModel structure: verify that all required properties (connections, listeners, services) are present and properly typed

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/**/*.{ts,tsx} : Define all constants (node types, sizing, spacing) in src/resources/constants.ts and import them where needed instead of hardcoding values

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/**/*.{ts,tsx} : Use TypeScript 5.8.3 with strict type checking enabled for all source files

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
📚 Learning: 2025-11-26T06:35:19.217Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:173-178
Timestamp: 2025-11-26T06:35:19.217Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, the commented-out debugging block in the verifyFileContent function (lines 172-177 containing console.log, page.pause, and updateDataFileSync) is intentionally kept as a developer utility for updating test data files when needed. It should not be removed.

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts
📚 Learning: 2025-11-20T11:04:33.712Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 898
File: workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx:38-38
Timestamp: 2025-11-20T11:04:33.712Z
Learning: In workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx, the use of `useDMQueryClausesPanelStore.getState()` to access `clauseToAdd` and `setClauseToAdd` (instead of the hook subscription pattern) is intentional to prevent re-renders when these state values change.

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
  • workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx
  • workspaces/ballerina/data-mapper/src/components/DataMapper/DataMapperEditor.tsx
  • workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClauseEditor.tsx
  • workspaces/ballerina/data-mapper/src/utils/DataMapperContext/DataMapperContext.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
  • workspaces/ballerina/ballerina-core/src/interfaces/data-mapper.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/DiagramContext.tsx : Use DiagramContext (React Context API) for state management across the diagram component, including expandedNodes and graphQLGroupOpen state

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/**/*.tsx : Use React 18.2.0 features including concurrent rendering and automatic batching; avoid class components in favor of functional components with hooks

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/index.ts : Export the main Diagram component as the default export from the package entry point (index.ts)

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/Diagram.tsx : Implement proper error boundaries around the Diagram component to gracefully handle rendering errors without crashing the parent application

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/nodes/**/*.{ts,tsx} : Each node type must have a Model class extending NodeModel, a Factory class implementing the factory pattern, and a Widget React component for visual representation

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/**/diagram.ts : Register custom factories with the diagram engine: NodePortFactory, NodeLinkFactory, ListenerNodeFactory, EntryNodeFactory, ConnectionNodeFactory, and OverlayLayerFactory

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/Diagram.tsx : Debounce diagram repaints to avoid excessive re-renders when state changes occur

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/nodes/**/!(*Widget|*Model).tsx : Implement the factory pattern for node instantiation: each node type (Listener, Entry, Connection) must have a corresponding Factory class that extends the appropriate factory interface

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/utils/diagram.ts : Create function-level links instead of service-level links: each function should connect individually to its target connections, not the entire service

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts
  • workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts
📚 Learning: 2025-11-26T06:37:07.886Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/dm-data/array-inner/inline/map3.bal.txt:6-8
Timestamp: 2025-11-26T06:37:07.886Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/dm-data/ directories, BAL test data files (such as those in array-inner/inline/, array-root/inline/, basic/inline/, and their reusable counterparts) may intentionally contain type errors and other violations. These are comparison files used to test data mapper functionality and error handling, so such errors should not be flagged as issues.

Applied to files:

  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-20T06:44:07.498Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 969
File: workspaces/ballerina/data-mapper/src/components/Diagram/utils/node-utils.ts:23-44
Timestamp: 2025-11-20T06:44:07.498Z
Learning: In the data mapper node-utils.ts, when `lastViewIndex == 0`, the code is at the root view and should NOT use `focusInputRootMap` for fallback input node resolution. The truthiness check on `lastViewIndex` is intentional to skip the fallback at the root level.

Applied to files:

  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/nodes/EntryNode/components/GraphQLServiceWidget.tsx : For GraphQL services, group functions by type (Query, Subscription, Mutation) and allow independent expand/collapse of each group. Default state: Query group open, Subscription/Mutation groups collapsed

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClauseEditor.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/nodes/EntryNode/components/**/*.tsx : Implement lazy expansion of functions: only render function items when they are visible to improve performance for large function lists

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/nodes/EntryNode/components/GeneralWidget.tsx : For regular services with more than 3 functions, show only the first 2 functions with a 'View All' button; for services with 3 or fewer functions, show all functions

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/resources/icons/**/*.tsx : Create separate SVG icon components in src/resources/icons/ for all diagram icons and import them as React components

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/**/*.tsx : Use Emotion for CSS-in-JS styling instead of inline styles or external CSS files

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
📚 Learning: 2025-11-24T14:51:49.267Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 998
File: workspaces/ballerina/data-mapper/src/components/DataMapper/Header/ExpressionBar.tsx:113-132
Timestamp: 2025-11-24T14:51:49.267Z
Learning: In workspaces/ballerina/data-mapper/src/components/DataMapper/Header/ExpressionBar.tsx, if `textFieldRef.current` is not undefined, `textFieldRef.current.inputElement` is guaranteed to exist. If `inputElement` doesn't exist when `current` exists, it's a fatal error that should reach the error boundary rather than being handled with defensive null checks.

Applied to files:

  • workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
🧬 Code graph analysis (8)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts (1)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/type-utils.ts (1)
  • isPrimitive (67-75)
workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts (1)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts (1)
  • mapSeqToX (191-228)
workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts (1)
workspaces/ballerina/ballerina-core/src/interfaces/data-mapper.ts (1)
  • IOType (86-104)
workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx (1)
workspaces/ballerina/data-mapper/src/store/store.ts (1)
  • useDMQueryClausesPanelStore (171-180)
workspaces/ballerina/data-mapper/src/components/DataMapper/DataMapperEditor.tsx (3)
workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/rpc-manager.ts (3)
  • addClauses (172-192)
  • mapWithCustomFn (303-323)
  • mapWithTransformFn (366-386)
workspaces/ballerina/ballerina-extension/src/core/extended-language-client.ts (3)
  • addClauses (813-815)
  • mapWithCustomFn (833-835)
  • mapWithTransformFn (837-839)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts (2)
  • mapWithCustomFn (143-146)
  • mapWithTransformFn (148-151)
workspaces/ballerina/data-mapper/src/utils/DataMapperContext/DataMapperContext.ts (1)
workspaces/ballerina/ballerina-core/src/interfaces/data-mapper.ts (4)
  • IntermediateClause (220-223)
  • Mapping (106-118)
  • FnMetadata (234-238)
  • IOType (86-104)
workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx (5)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts (5)
  • createNewMapping (30-69)
  • mapWithQuery (153-177)
  • mapWithCustomFn (143-146)
  • mapWithTransformFn (148-151)
  • mapSeqToX (191-228)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts (1)
  • genArrayElementAccessSuffix (159-167)
workspaces/common-libs/ui-toolkit/src/components/Codicon/Codicon.tsx (1)
  • Codicon (44-56)
workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts (1)
  • InputOutputPortModel (55-129)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/type-utils.ts (3)
  • isNumericType (98-103)
  • getGenericTypeKind (77-96)
  • isPrimitive (67-75)
workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts (2)
workspaces/ballerina/data-mapper/src/utils/DataMapperContext/DataMapperContext.ts (1)
  • IDataMapperContext (21-37)
workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts (1)
  • InputOutputPortModel (55-129)
🔇 Additional comments (37)
workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts (2)

603-640: LGTM! Well-structured state preservation for GROUP_BY handling.

The save-and-restore pattern for groupById correctly scopes the grouping context during recursive type processing, preventing state leakage across sibling branches.


735-762: LGTM! Correct handling of sequence vs grouping key distinction.

The logic properly sets isSeq to false for the grouping key field while marking other fields as sequences when in a group-by context. This enables correct UI rendering for grouped data.

workspaces/ballerina/ballerina-core/src/interfaces/data-mapper.ts (3)

63-63: LGTM! Well-structured additions for GROUP_BY support.

The new GROUP_BY clause type, isSeq flags, and groupById tracking field provide a consistent foundation for group-by functionality across the data mapper.

Also applies to: 99-99, 146-146, 181-181


60-60: Verify the ORDER_BY value change from "order by" to "order-by" with manual inspection.

The identified breaking change is valid: this string value change will break any code comparing against the old "order by" literal. However, automated verification could not complete due to infrastructure limitations. Manual verification is required to:

  • Search the codebase and tests for references to the old "order by" string value
  • Check if any backend systems or serialized data depend on this value
  • Ensure all consumers are updated consistently before merging

199-199: I'm encountering persistent network issues accessing the repository. However, based on the review comment provided, I can see the verification request is well-founded:

The comment correctly identifies a structural type change where fromClause migrates from a flat FromClause interface to a nested IntermediateClause structure. The comment also provides evidence (utils.ts line 615) showing the new access pattern: model.query!.fromClause.properties.name.

Since I cannot access the codebase to verify whether all consumers have been updated to use the new nested access pattern (.properties.* instead of direct property access), I must mark this for manual verification.


Verify: fromClause type changed to IntermediateClause.

The fromClause property now uses IntermediateClause (with type and properties sub-structure) instead of the flat FromClause interface. Ensure all consumers access properties via fromClause.properties.name, fromClause.properties.expression, etc. Verify there are no remaining direct accesses like fromClause.name or fromClause.expression that would break with this type change.

workspaces/ballerina/data-mapper/src/components/Diagram/utils/type-utils.ts (1)

31-34: LGTM! Clear sequence type display with ... prefix.

The "..." prefix provides a clear visual indicator for sequence types in the UI, following common language conventions for variadic/spread notation.

workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx (3)

563-563: LGTM! Using TriggerKind enum improves readability.

Replacing magic numbers with TriggerKind.INVOKED and TriggerKind.TRIGGER_CHARACTER makes the code self-documenting.

Also applies to: 655-655


761-768: LGTM! Model signature now tracks LET and GROUP_BY clauses.

Including these intermediate clauses in the signature ensures proper change detection when clauses are added, removed, or modified.


742-742: LGTM!

The genUniqueName function is correctly wired to the DataMapper component for use in clause creation.

workspaces/ballerina/data-mapper/src/components/Diagram/utils/common-utils.ts (2)

68-68: LGTM! SeqToArray excluded from pending mapping requirement.

This correctly indicates that sequence-to-array mappings can be applied directly without requiring an intermediate pending state.


98-106: Unable to Complete Verification — Manual Check Required

I encountered persistent network errors while attempting to access the repository to verify whether the MappingType enum contains the SeqToPrimitive and SeqToArray constants referenced in the code (lines 98-106).

The sequence mapping logic shown appears sound in structure:

  • sourceField.isSeq check correctly guards sequence-specific mappings
  • isPrimitive(targetField.kind) and targetField.kind === TypeKind.Array conditions are semantically appropriate
  • Placement before dimension-based array checks ensures proper precedence

However, the approval is contingent on verifying that these mapping type constants actually exist in the MappingType enum. Please manually confirm:

  1. MappingType.SeqToPrimitive is defined in the MappingType enum
  2. MappingType.SeqToArray is defined in the MappingType enum
workspaces/ballerina/data-mapper/src/components/Diagram/Port/model/InputOutputPortModel.ts (2)

23-25: LGTM! Imports aligned with new sequence mapping functionality.

The added imports for mapSeqToX and DataMapperNodeModel support the new SeqToArray mapping behavior introduced below.


84-88: LGTM! SeqToArray mapping correctly delegates to mapSeqToX.

The new logic properly handles SeqToArray mappings by:

  • Extracting the target node context
  • Invoking mapSeqToX with a bracket-wrapping modifier [${expr}]
  • Returning early to bypass the default mapping creation flow

This aligns with the broader GROUP_BY sequence mapping support added in this PR.

workspaces/ballerina/data-mapper/src/index.tsx (1)

85-85: LGTM! Public API extended with genUniqueName.

The new genUniqueName prop enables asynchronous unique-name generation for GROUP_BY and other intermediate clauses. The signature is clear and consistent with the rest of the public API.

workspaces/ballerina/data-mapper/src/components/Diagram/Link/DataMapperLink/DataMapperLink.ts (1)

34-35: LGTM! MappingType enum extended for sequence mappings.

The new SeqToPrimitive and SeqToArray variants clearly extend the mapping type surface to support sequence-to-primitive and sequence-to-array transformations introduced in this PR.

workspaces/ballerina/data-mapper/src/components/Diagram/hooks/useDiagramModel.ts (3)

29-29: LGTM! IntermediateClauseType import supports GROUP_BY filtering.

The import enables filtering of GROUP_BY clauses in the queryClauses computation below.


51-53: LGTM! queryClauses computation enables GROUP_BY change detection.

The computation correctly:

  • Filters for LET and GROUP_BY intermediate clauses
  • Maps them to "name:expression" strings for change tracking
  • Will trigger diagram refetch when GROUP_BY clauses are added/modified

107-108: LGTM! queryClauses added to query key for reactive updates.

Adding queryClauses to the useQuery dependency array ensures the diagram model refetches when GROUP_BY or LET clauses change, maintaining UI consistency.

workspaces/ballerina/data-mapper/src/components/Diagram/utils/modification-utils.ts (3)

180-189: Verify: Is mapSeqToArray intentionally scaffolded?

The function has guards but no implementation. Confirm whether this is intentional scaffolding for future work or if implementation is missing for this PR.


191-228: LGTM! mapSeqToX correctly handles sequence mapping with GROUP_BY awareness.

The implementation:

  • Guards against missing ports (lines 192-196)
  • Creates a LET clause with unique name when source field is not focused (lines 200-212)
  • Intelligently positions the LET clause before any GROUP_BY clause (lines 214-217)
  • Updates field FQN references (lines 221-222)
  • Delegates to createNewMapping with the provided modifier (line 226)

The GROUP_BY-aware placement logic ensures correct clause ordering in the generated query.


232-235: LGTM! Defensive guard added to mapWithJoin.

The sourcePort null check prevents potential null-pointer errors before accessing the port model.

workspaces/ballerina/data-mapper/src/components/DataMapper/DataMapperEditor.tsx (3)

147-147: LGTM! genUniqueName prop properly destructured.

The new prop is correctly extracted from DataMapperEditorProps for use in component initialization.


240-245: LGTM! genUniqueName threaded through DataMapperContext.

The prop flow is correct:

  • Line 240: addClauses passed to context
  • Lines 244-245: enrichChildFields and genUniqueName passed in proper order

This enables unique name generation for GROUP_BY and other intermediate clauses throughout the data mapper flow.


370-370: LGTM! genUniqueName propagated to ClausesPanel.

The prop is correctly passed to ClausesPanel, enabling unique name generation in the GROUP_BY clause editor UI.

workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClauseEditor.tsx (4)

45-51: LGTM! Clause type options enhanced with clear labels and GROUP_BY support.

The localized labels ("Condition", "Local variable", "Sort by", etc.) improve usability, and the new "Group by" option extends the clause configuration UI as intended.


80-88: LGTM! GROUP_BY expression field labels are clear and contextual.

The conditional logic provides appropriate labels and documentation:

  • "Grouping Key" with "Enter the grouping key expression" for GROUP_BY
  • "Join With Collection" with "Collection to be joined" for JOIN
  • Generic "Expression" for other clause types

131-137: Verify: Does removing JOIN special handling break existing functionality?

The previous implementation explicitly set type: "var" and isOuter: false for JOIN clauses (lines removed). Now all clause types uniformly assign data to clause.properties.

Confirm that:

  1. The JOIN form fields now properly collect these values, OR
  2. The backend no longer requires these fields to be explicitly set

155-156: LGTM! GROUP_BY clause requires only expression field.

Returning only [expressionField] for GROUP_BY is correct, as grouping clauses don't require name or type fields like LET/FROM clauses.

workspaces/ballerina/data-mapper/src/utils/DataMapperContext/DataMapperContext.ts (4)

18-18: LGTM! IntermediateClause import supports new addClauses method.

The import enables the addClauses method signature added to the interface below.


31-31: LGTM! addClauses method added to IDataMapperContext.

The method signature correctly supports adding intermediate clauses (including GROUP_BY) with proper parameters for target field, new/existing flag, and insertion index.


36-36: LGTM! genUniqueName method added to IDataMapperContext.

The async method signature enables unique name generation for GROUP_BY and other clauses based on a base name and viewId.


51-56: LGTM! Constructor parameters updated consistently.

The constructor parameters correctly include:

  • addClauses at line 51
  • enrichChildFields and genUniqueName at lines 55-56

The implementation matches the interface definition, maintaining consistency across the DataMapperContext API.

workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx (2)

26-26: LGTM!

Import and prop additions are clean. The genUniqueName prop signature correctly reflects the async nature of the unique name generation.

Also applies to: 35-35, 41-41


61-66: LGTM!

The fillDefaults integration correctly awaits the async operation before proceeding with addClauses.

workspaces/ballerina/data-mapper/src/components/Diagram/Label/MappingOptionsWidget.tsx (3)

109-139: LGTM!

The onClick handlers are well-structured and appropriately scoped within the useMemo callback since they close over link and context.


174-192: LGTM!

The aggregate function generation correctly filters available functions based on the source port's type kind, with appropriate fallbacks.


219-251: Verify SeqToArray handling in genMenuItems.

MappingType.SeqToArray is checked in the condition at line 238 to exclude custom/transform function options, but it's not explicitly handled in the genMenuItems switch statement. It falls through to defaultMenuItems ("Map Anyway").

Based on the relevant code snippet from InputOutputPortModel.ts (lines 82-85), SeqToArray is handled directly in createLinkModel via mapSeqToX, so this widget might not need a specific case. Please verify this is the intended behavior.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx (1)

365-382: Returning { line: 0, offset: 0 } on error may silently fail.

The catch block returns a fallback position that might be interpreted as valid by callers, hiding the actual error. Consider either propagating the error or returning a Result-like type that explicitly signals failure.

Apply this diff to propagate the error:

     } catch (error) {
-        console.error(error);
-        return { line: 0, offset: 0  };
+        console.error("Failed to get clause position:", error);
+        throw error;
     }

Alternatively, if callers need to handle missing positions gracefully, return a nullable type:

-const getClausePosition = async (targetField: string, index: number) => {
+const getClausePosition = async (targetField: string, index: number): Promise<NodePosition | null> => {
     try {
         // ... existing code ...
     } catch (error) {
         console.error(error);
-        return { line: 0, offset: 0  };
+        return null;
     }
 }
🧹 Nitpick comments (2)
workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts (1)

634-636: Consider try-finally for state restoration.

The state restoration for model.groupById could be more robust by using a try-finally block to ensure cleanup even if exceptions occur during processing.

Apply this diff to improve state management:

     let isGroupByIdUpdated = false;
     const prevGroupById = model.groupById;

     if (model.focusInputs) {
         const focusMember = model.focusInputs[parentId];
         if (focusMember) {
             member = focusMember;
             parentId = member.name;
             fieldId = member.name;
             isFocused = true;
             model.focusInputRootMap[fieldId] = model.traversingRoot;

             if (model.query?.fromClause.properties.name === fieldId) {
                 const groupByClause = model.query.intermediateClauses?.find(clause => clause.type === IntermediateClauseType.GROUP_BY);
                 if (groupByClause) {
                     model.groupById = groupByClause.properties.name;
                     isGroupByIdUpdated = true;
                 }
             }
         }
     }

     const ioType: IOType = {
         id: parentId,
         name: member.name,
         displayName: member.displayName,
         typeName: member.typeName!,
         kind: member.kind,
         ...(isFocused && { isFocused }),
         ...(member.optional !== undefined && { optional: member.optional }),
         ...(member.typeInfo && { typeInfo: member.typeInfo })
     };

-    const typeSpecificProps = processTypeKind(member, parentId, model, visitedRefs);
-
-    if(isGroupByIdUpdated){
-        model.groupById = prevGroupById;
-    }
-
-    return {
-        ...ioType,
-        ...typeSpecificProps
-    };
+    try {
+        const typeSpecificProps = processTypeKind(member, parentId, model, visitedRefs);
+        return {
+            ...ioType,
+            ...typeSpecificProps
+        };
+    } finally {
+        if (isGroupByIdUpdated) {
+            model.groupById = prevGroupById;
+        }
+    }
workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx (1)

771-782: Consider defensive string interpolation for clause properties.

The clause mapping creates a signature string by interpolating type, name, and expression properties. While form validation enforces these as required, consider using nullish coalescing for robustness:

Apply this diff for more defensive string building:

     ...(model.query?.intermediateClauses
         ?.filter((clause) => (clause.type === IntermediateClauseType.LET || clause.type === IntermediateClauseType.GROUP_BY))
-        .map(clause => `${clause.properties.type} ${clause.properties.name} ${clause.properties.expression}`)
+        .map(clause => `${clause.properties.type ?? ''} ${clause.properties.name ?? ''} ${clause.properties.expression ?? ''}`)
         || [])

Based on learnings, clause.properties.expression is enforced by form validation, so this is optional defensive coding.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between df42e40 and 79c8e2d.

📒 Files selected for processing (2)
  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts (4 hunks)
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx (7 hunks)
🧰 Additional context used
🧠 Learnings (10)
📚 Learning: 2025-11-26T06:37:07.886Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/dm-data/array-inner/inline/map3.bal.txt:6-8
Timestamp: 2025-11-26T06:37:07.886Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/dm-data/ directories, BAL test data files (such as those in array-inner/inline/, array-root/inline/, basic/inline/, and their reusable counterparts) may intentionally contain type errors and other violations. These are comparison files used to test data mapper functionality and error handling, so such errors should not be flagged as issues.

Applied to files:

  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-26T06:35:19.217Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:173-178
Timestamp: 2025-11-26T06:35:19.217Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, the commented-out debugging block in the verifyFileContent function (lines 172-177 containing console.log, page.pause, and updateDataFileSync) is intentionally kept as a developer utility for updating test data files when needed. It should not be removed.

Applied to files:

  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
📚 Learning: 2025-11-20T06:44:07.498Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 969
File: workspaces/ballerina/data-mapper/src/components/Diagram/utils/node-utils.ts:23-44
Timestamp: 2025-11-20T06:44:07.498Z
Learning: In the data mapper node-utils.ts, when `lastViewIndex == 0`, the code is at the root view and should NOT use `focusInputRootMap` for fallback input node resolution. The truthiness check on `lastViewIndex` is intentional to skip the fallback at the root level.

Applied to files:

  • workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts
📚 Learning: 2025-11-27T07:59:33.522Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 897
File: workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx:50-59
Timestamp: 2025-11-27T07:59:33.522Z
Learning: In workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx, the `clause.properties.expression` property in the `fillDefaults` function does not require defensive null/undefined checks because it's a required property enforced by form validation in ClauseEditor.

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-20T11:04:33.712Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 898
File: workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx:38-38
Timestamp: 2025-11-20T11:04:33.712Z
Learning: In workspaces/ballerina/data-mapper/src/components/DataMapper/SidePanel/QueryClauses/ClausesPanel.tsx, the use of `useDMQueryClausesPanelStore.getState()` to access `clauseToAdd` and `setClauseToAdd` (instead of the hook subscription pattern) is intentional to prevent re-renders when these state values change.

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-25T06:34:10.812Z
Learnt from: CR
Repo: wso2/vscode-extensions PR: 0
File: workspaces/ballerina/component-diagram/AGENTS.md:0-0
Timestamp: 2025-11-25T06:34:10.812Z
Learning: Applies to workspaces/ballerina/component-diagram/src/components/**/*.tsx : Use React 18.2.0 features including concurrent rendering and automatic batching; avoid class components in favor of functional components with hooks

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-24T14:51:49.267Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 998
File: workspaces/ballerina/data-mapper/src/components/DataMapper/Header/ExpressionBar.tsx:113-132
Timestamp: 2025-11-24T14:51:49.267Z
Learning: In workspaces/ballerina/data-mapper/src/components/DataMapper/Header/ExpressionBar.tsx, if `textFieldRef.current` is not undefined, `textFieldRef.current.inputElement` is guaranteed to exist. If `inputElement` doesn't exist when `current` exists, it's a fatal error that should reach the error boundary rather than being handled with defensive null checks.

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-26T06:34:09.752Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:128-134
Timestamp: 2025-11-26T06:34:09.752Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, the goPrevViewBreadcrumb() method is only called when in a focused view, ensuring breadcrumbs are always present. No guard for empty breadcrumbs is needed.

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-26T07:49:56.428Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:136-141
Timestamp: 2025-11-26T07:49:56.428Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, the goPrevViewBackButton() method is only called when in a focused view, ensuring breadcrumbs are always present. No guard for empty breadcrumbs is needed.

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
📚 Learning: 2025-11-26T06:33:22.950Z
Learnt from: KCSAbeywickrama
Repo: wso2/vscode-extensions PR: 653
File: workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts:86-112
Timestamp: 2025-11-26T06:33:22.950Z
Learning: In workspaces/bi/bi-extension/src/test/e2e-playwright-tests/data-mapper/DataMapperUtils.ts, empty catch blocks around progress ring waitForSelector calls (state: 'attached' and 'detached') are intentional. The progress ring duration depends on machine performance and may appear very briefly, causing the wait to miss the event. The try-catch allows the test to proceed gracefully regardless of timing.

Applied to files:

  • workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx
🔇 Additional comments (7)
workspaces/ballerina/ballerina-extension/src/rpc-managers/data-mapper/utils.ts (4)

34-35: LGTM! Imports align with GROUP_BY clause support.

The new imports are properly utilized for the group-by functionality introduced in this PR.


731-746: LGTM! Focus and sequence detection logic is correct.

The implementation properly tracks focused fields and sequence status for group-by operations. The logic correctly excludes the groupById field itself from being marked as a sequence.

Minor style note: Consider consistent spacing in if statements (line 742).


754-755: LGTM! Conditional property spreading aligns with IOType interface.

The pattern correctly adds isFocused and isSeq properties only when truthy, matching the optional nature of these properties in the IOType interface.


599-617: I apologize for the repository access issue. Unfortunately, I'm unable to clone the repository in the sandbox environment, which prevents me from directly verifying the code and type definitions.

However, based on the learnings from previous reviews in this same repository, I can provide important context:

From prior reviews in wso2/vscode-extensions:

  1. In ExpressionBar.tsx (lines 113-132): The codebase follows a pattern where if a parent object exists, its guaranteed child properties should not require defensive null checks — they should be accessed directly.
  2. In ClausesPanel.tsx (lines 50-59): Properties enforced by design or validation (like clause.properties.expression) do not require defensive null/undefined checks.

This suggests the codebase philosophy is to trust guaranteed properties rather than add redundant defensive coding.

However, without repository access, I cannot verify:

  • Whether IOTypeField actually has an isSeq property
  • The exact type of the member variable at line 611
  • Whether model.query is truly non-null at that execution point
  • The broader control flow guarantees

Given this limitation and the complexity of the claims, I need to mark this for manual verification:


Verify null assertion pattern and isSeq property access in context of codebase design patterns.

The review raises concerns about non-null assertions and property access that require verification against the actual type definitions and control flow guarantees in the codebase. Based on learnings from previous reviews in this repository, properties may be guaranteed by design, but this needs confirmation for the specific model.query and member.isSeq access patterns at lines 611-617.

workspaces/ballerina/ballerina-visualizer/src/views/DataMapper/DataMapperView.tsx (3)

46-48: LGTM!

The new imports are correctly used throughout the file: IORoot for type annotations, IntermediateClauseType for clause filtering, and TriggerKind for completion trigger types.


752-752: I'm unable to complete the verification due to persistent repository access issues in the sandbox environment. However, I can address the review comment by noting that the script provided contains a syntax error and improving the overall clarity.

Fix the script syntax error and verify DataMapper prop types directly in the IDE.

The provided script contains a typo: ast-greg should be ast-grep on line 10. Additionally, search the DataMapper component definition (likely in workspaces/ballerina/ballerina-visualizer/src/components/DataMapper/ or similar location) to confirm its prop interface includes genUniqueName with the signature (name: string, viewId: string) => Promise<string>.


550-584: Verification incomplete due to repository access failure; manual verification needed for TriggerKind.INVOKED appropriateness.

The web search confirms that TriggerKind.INVOKED is a standard LSP CompletionTriggerKind for manual invocation and normal identifier typing. However, I was unable to access the repository to verify:

  • Whether other code patterns in the codebase use different trigger kinds for scope-wide completions
  • The actual available trigger kinds in Ballerina's RPC implementation
  • Whether an empty expression context with INVOKED correctly retrieves all available names in scope

The undefined property guard (lines 557-560) correctly addresses the previous TODO concern by returning the original name as a fallback. The unique-name generation logic is sound. However, confirming whether TriggerKind.INVOKED is the optimal trigger kind for this scope-wide completion request requires manual verification with the Ballerina LSP documentation or similar code patterns in the codebase.

@KCSAbeywickrama KCSAbeywickrama merged commit 0a51b50 into wso2:main Nov 27, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Group by clause support for the Data mapper in frontend UI

2 participants