Skip to content

[Grida Canvas] Fix Clipboard IO #375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 31, 2025
Merged

[Grida Canvas] Fix Clipboard IO #375

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
880fdc5
chore
softmarshmallow May 31, 2025
7a31aa0
add images repo
softmarshmallow May 31, 2025
1772f91
add create node methods
softmarshmallow May 31, 2025
67c102d
clean editor method usage
softmarshmallow May 31, 2025
ff23983
improve clipboard io model with plain text
softmarshmallow May 31, 2025
cf0f2ac
fix paste issue on safari
softmarshmallow May 31, 2025
b488234
chore
softmarshmallow May 31, 2025
b676264
fix non resolving
softmarshmallow May 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions editor/app/(dev)/canvas/tools/io-svg/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,13 +93,12 @@ export default function IOSVGPage() {
)
);

instance.dispatch({
type: "__internal/reset",
state: editor.state.init({
instance.reset(
editor.state.init({
editable: true,
document: doc,
}),
});
})
);
}
setResult(result);
//
Expand Down
9 changes: 4 additions & 5 deletions editor/components/formfield/grida-canvas/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,14 +70,13 @@ export function GridaCanvasFormField() {
useEffect(() => {
fetch("/examples/canvas/sketch-teimplate-01.grida").then((res) => {
res.json().then((file) => {
instance.dispatch({
type: "__internal/reset",
key: "template",
state: editor.state.init({
instance.reset(
editor.state.init({
editable: true,
document: file.document,
}),
});
"template"
);
});
});
}, []);
Expand Down
253 changes: 127 additions & 126 deletions editor/grida-canvas-react/provider.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1025,7 +1025,10 @@ export function useDataTransferEventTarget() {
viewportdomrect.y,
];
const translate = cmath.transform.getTranslate(state.transform);
return cmath.vector2.sub(xy, viewport_pos, translate);
return cmath.vector2.quantize(
cmath.vector2.sub(xy, viewport_pos, translate),
1
);
},
[state.transform]
);
Expand All @@ -1042,21 +1045,17 @@ export function useDataTransferEventTarget() {
position ? [position.clientX, position.clientY] : [0, 0]
);

const node = {
type: "text",
text: text,
width: "auto",
height: "auto",
left: x,
top: y,
} satisfies grida.program.nodes.NodePrototype;
instance.insertNode(node);
const node = instance.createTextNode();
node.$.name = text;
node.$.text = text;
node.$.left = x;
node.$.top = y;
},
Comment on lines +1048 to 1053
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add error handling for node creation methods.

The new node creation methods might fail, but there's no error handling.

 const insertText = useCallback(
   (
     text: string,
     position?: {
       clientX: number;
       clientY: number;
     }
   ) => {
     const [x, y] = canvasXY(
       position ? [position.clientX, position.clientY] : [0, 0]
     );

-    const node = instance.createTextNode();
-    node.$.name = text;
-    node.$.text = text;
-    node.$.left = x;
-    node.$.top = y;
+    try {
+      const node = instance.createTextNode();
+      if (!node) {
+        throw new Error('Failed to create text node');
+      }
+      node.$.name = text;
+      node.$.text = text;
+      node.$.left = x;
+      node.$.top = y;
+    } catch (error) {
+      toast.error('Failed to insert text');
+      console.error(error);
+    }
   },
   [instance, canvasXY]
 );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const node = instance.createTextNode();
node.$.name = text;
node.$.text = text;
node.$.left = x;
node.$.top = y;
},
const insertText = useCallback(
(
text: string,
position?: {
clientX: number;
clientY: number;
}
) => {
const [x, y] = canvasXY(
position ? [position.clientX, position.clientY] : [0, 0]
);
try {
const node = instance.createTextNode();
if (!node) {
throw new Error('Failed to create text node');
}
node.$.name = text;
node.$.text = text;
node.$.left = x;
node.$.top = y;
} catch (error) {
toast.error('Failed to insert text');
console.error(error);
}
},
[instance, canvasXY]
);
🤖 Prompt for AI Agents 
In editor/grida-canvas-react/provider.tsx around lines 1048 to 1053, the code
creates a text node without any error handling, which could cause unhandled
exceptions if node creation fails. Wrap the node creation and property
assignments in a try-catch block to catch any errors, and handle them
appropriately, such as logging the error or returning a fallback value, to
ensure the application remains stable.

[instance, canvasXY]
);

const insertImage = useCallback(
(
async (
name: string,
file: File,
position?: {
Expand All @@ -1069,93 +1068,62 @@ export function useDataTransferEventTarget() {
);

// TODO: uploader is not implemented. use uploader configured by user.

const url = URL.createObjectURL(file);
const img = new Image();
const createAndInsertNode = (width: number, height: number) => {
const node = {
type: "image",
name: name,
width,
height,
fit: "cover",
src: url,
left: x,
top: y,
} satisfies grida.program.nodes.NodePrototype;
instance.insertNode(node);
};

img.onload = () => {
createAndInsertNode(img.naturalWidth, img.naturalHeight);
};
img.onerror = () => {
createAndInsertNode(100, 100);
};
img.src = url;
const image = await instance.createImage(url);
const node = instance.createImageNode(image);
node.$.position = "absolute";
node.$.name = name;
node.$.left = x;
node.$.top = y;
},
[instance, canvasXY]
);

const insertSVG = useCallback(
(
async (
name: string,
svg: string,
position?: {
clientX: number;
clientY: number;
}
) => {
const optimized = iosvg.v0.optimize(svg).data;
iosvg.v0
.convert(optimized, {
name: name,
currentColor: { r: 0, g: 0, b: 0, a: 1 },
})
.then((result) => {
if (result) {
result = result as grida.program.nodes.i.IPositioning &
grida.program.nodes.i.IFixedDimension;

const center_dx =
typeof result.width === "number" ? result.width / 2 : 0;

const center_dy =
typeof result.height === "number" ? result.height / 2 : 0;

const [x, y] = canvasXY(
cmath.vector2.sub(
position ? [position.clientX, position.clientY] : [0, 0],
[center_dx, center_dy]
)
);
const node = await instance.createNodeFromSvg(svg);

result.left = x;
result.top = y;
instance.insertNode(result);
} else {
throw new Error("Failed to convert SVG");
}
});
const center_dx =
typeof node.$.width === "number" && node.$.width > 0
? node.$.width / 2
: 0;

const center_dy =
typeof node.$.height === "number" && node.$.height > 0
? node.$.height / 2
: 0;

const [x, y] = canvasXY(
cmath.vector2.sub(
position ? [position.clientX, position.clientY] : [0, 0],
[center_dx, center_dy]
)
);

node.$.name = name;
node.$.left = x;
node.$.top = y;
},
[instance, canvasXY]
);

const insertFromFile = useCallback(
(
type: io.clipboard.ValidFileType,
file: File,
position?: {
clientX: number;
clientY: number;
}
) => {
const type = file.type || file.name.split(".").pop() || file.name;
const is_svg = type === "image/svg+xml";
const is_png = type === "image/png";
const is_jpg = type === "image/jpeg";
const is_gif = type === "image/gif";

if (is_svg) {
if (type === "image/svg+xml") {
const reader = new FileReader();
reader.onload = (e) => {
const svgContent = e.target?.result as string;
Expand All @@ -1164,21 +1132,29 @@ export function useDataTransferEventTarget() {
};
reader.readAsText(file);
return;
}

if (is_png || is_jpg || is_gif) {
} else if (
type === "image/png" ||
type === "image/jpeg" ||
type === "image/gif"
) {
const name = file.name.split(".")[0];
insertImage(name, file, position);
return;
}

toast.error(`insertion of file type ${type} is not supported`);
},
[insertImage, insertSVG]
);

/**
* pasting from os clipboard (or fallbacks to local clipboard)
*
* 1. if the payload contains valid grida payload, insert it (or if identical to local clipboard, paste it)
* 2. if the payload contains text/plain, image/png, image/jpeg, image/gif, image/svg+xml, insert it
* 3. if the payload contains no valid payload, fallback to local clipboard, and paste it
*
*/
const onpaste = useCallback(
(event: ClipboardEvent) => {
async (event: ClipboardEvent) => {
if (event.defaultPrevented) return;
// cancel if on contenteditable / form element
if (
Expand All @@ -1194,58 +1170,79 @@ export function useDataTransferEventTarget() {
return;
}

const items = event.clipboardData.items;

let pasted_from_data_transfer = false;
for (let i = 0; i < items.length; i++) {
const item = items[i];
if (item.kind === "file") {
const file = item.getAsFile();
if (file) {
insertFromFile(file, {
clientX: window.innerWidth / 2,
clientY: window.innerHeight / 2,
});
pasted_from_data_transfer = true;
}
} else if (item.kind === "string" && item.type === "text/plain") {

// NOTE: the read of the clipboard data should be non-blocking. (in safari, this will fail without any error)
const items = (
await Promise.all(
Array.from(event.clipboardData.items).map(async (item) => {
try {
const payload = await io.clipboard.decode(item);
return payload;
} catch {
return null;
}
})
)
).filter((item) => item !== null);

const grida_payload = items.find((item) => item.type === "clipboard");

// 1. if there is a grida html clipboard, use it and ignore all others.
if (grida_payload) {
if (
current_clipboard?.payload_id === grida_payload.clipboard.payload_id
) {
instance.paste();
pasted_from_data_transfer = true;
item.getAsString((data) => {
insertText(data, {
clientX: window.innerWidth / 2,
clientY: window.innerHeight / 2,
});
} else {
grida_payload.clipboard.prototypes.forEach((p) => {
const sub =
grida.program.nodes.factory.create_packed_scene_document_from_prototype(
p,
nid
);
instance.insert({ document: sub });
});
} else if (item.kind === "string" && item.type === "text/html") {
pasted_from_data_transfer = true;
item.getAsString((html) => {
const data = io.clipboard.decodeClipboardHtml(html);
if (data) {
if (current_clipboard?.payload_id === data.payload_id) {
instance.paste();
} else {
data.prototypes.forEach((p) => {
const sub =
grida.program.nodes.factory.create_packed_scene_document_from_prototype(
p,
nid
);
instance.insert({ document: sub });
}
}
// 2. if the payload contains text/plain, image/png, image/jpeg, image/gif, image/svg+xml, insert it
else {
for (let i = 0; i < items.length; i++) {
const item = items[i];
try {
switch (item.type) {
case "text": {
const { text } = item;
insertText(text, {
clientX: window.innerWidth / 2,
clientY: window.innerHeight / 2,
});
pasted_from_data_transfer = true;
break;
}
case "image/gif":
case "image/jpeg":
case "image/png":
case "image/svg+xml": {
const { type, file } = item;
insertFromFile(type, file, {
clientX: window.innerWidth / 2,
clientY: window.innerHeight / 2,
});
pasted_from_data_transfer = true;
break;
}
return;
}
insertText(html, {
clientX: window.innerWidth / 2,
clientY: window.innerHeight / 2,
});
});
} catch {}
}
}

// 3. if the payload contains no valid payload, fallback to local clipboard, and paste it
if (!pasted_from_data_transfer) {
event.preventDefault();
instance.paste();
event.preventDefault();
}
},
[instance, insertFromFile, insertText, current_clipboard]
Expand Down Expand Up @@ -1289,7 +1286,12 @@ export function useDataTransferEventTarget() {
const files = event.dataTransfer.files;
for (let i = 0; i < files.length; i++) {
const file = files[i];
insertFromFile(file, event);
const [valid, type] = io.clipboard.filetype(file);
if (valid) {
insertFromFile(type, file, event);
} else {
toast.error(`file type '${type}' is not supported`);
}
}
},
[insertFromFile]
Expand All @@ -1309,18 +1311,17 @@ export function useClipboardSync() {
useEffect(() => {
try {
if (user_clipboard) {
const htmltxt = io.clipboard.encodeClipboardHtml(
const items = io.clipboard.encode(
user_clipboard as io.clipboard.ClipboardPayload
);
const blob = new Blob([htmltxt], { type: "text/html" });

const clipboardItem = new ClipboardItem({
"text/html": blob,
});
navigator.clipboard.write([clipboardItem]);
if (items) {
const clipboardItem = new ClipboardItem(items);
navigator.clipboard.write([clipboardItem]);
}
}
} catch (e) {
//
reportError(e);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Import or define the reportError function.

The reportError function is used but not imported, which will cause a runtime error.

🏁 Script executed:

#!/bin/bash
# Check if reportError is defined or imported elsewhere
rg -A 5 "reportError" --type ts --type tsx

Length of output: 70

Let’s broaden the search across all script and markup file types to be sure:

🏁 Script executed:

#!/bin/bash
# Search for any occurrence or definition of reportError in TS, TSX, JS, and JSX files
rg -Hn --glob "*.{ts,tsx,js,jsx}" -A5 "reportError" .

Length of output: 7563

Define or import reportError in provider.tsx

The call to reportError(e) on line 1303 of editor/grida-canvas-react/provider.tsx has no matching definition or import anywhere in the codebase, which will lead to a runtime error.

Please add the appropriate import or define reportError before using it:

• File: editor/grida-canvas-react/provider.tsx
Line 1303:

- reportError(e);
+ reportError(e); // ← ensure `reportError` is imported or defined

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In editor/grida-canvas-react/provider.tsx at line 1303, the function reportError
is called but not imported or defined, causing a runtime error. To fix this,
locate the correct module that exports reportError and add an import statement
for it at the top of provider.tsx, or define the reportError function locally if
it does not exist elsewhere. Ensure the import or definition is placed before
line 1303 so the function is available when called.

}
}, [user_clipboard]);
//
Expand Down
Loading